DOCSLIB.ORG

Utilization and Deployment of Software Testing Tools and Techniques

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

International Journal of Trend in Research and Development, Volume 6(1), ISSN: 2394-9333 www.ijtrd.com Utilization and Deployment of Software Testing Tools and Techniques Naw Tin Tin Yu University of Computer Studies (Hpa-an), Kayin State, Myanmar Abstract: Software testing is a process of executing a program design and implementation, maintenance, process and or application with the intent of finding the software bugs. management issues in softwareengineering. Our study focuses Software testing is an activity to check whether the actual on the state of the art in testing techniques, as well as the latest results match the expected results and to ensure that the techniques which representing the future direction of this area. software system is Defect free. It involves execution of a Software testing techniques are based in an amalgam of software component or system component to evaluate one or methods drawn from graphtheory, programming language, more properties of interest. Software testing also helps to reliability assessment, reliable-testing theory, etc.[12]. identify errors, gaps or missing requirements in contrary to the Software testing is very important because of the following actual requirements. Software testing tool is used in day to reasons: Software testing is really required to point out the days testing activities.In this paper, testing tools and techniques defects and errors that were made during the development have been described. Software testing is gaining more and phases. It‟s essential since it makes sure of the Customer‟s more importance in the future. reliability and their satisfaction in the application. It is very important to ensure the Quality of the product. Quality product Keywords: Software testing, Tools, Techniques. delivered to the customers helps in gaining their confidence. I. INTRODUCTION (Know more about Software Quality). Testing is necessary in order to provide the facilities to the customers like the delivery Software testing is an investigation conducted to provide of high quality product or software application which requires stakeholders with information about the quality of the software lower maintenance cost and hence results into more accurate, product or service under test. Software testing can also provide consistent and reliable results. Testing is required for an an objective, independent view of the software to allow the effective performance of software application or product. It‟s business to appreciate and understand the risks of software important to ensure that the application should not result into implementation. Test techniques include the process of any failures because it can be very expensive in the future or in executing a program or application with the intent of finding the later stages of the development. It‟s required to stay in the software bugs (errors or other defects), and verifying that the business. [5]Testing improves the quality of the final product software product is fit for use [2]. Testing is an important because most of the errors found get corrected before the technique to check and control software quality. It is partof software is released into production. Test tool selection is a almost any software development project. Time and resources very important part of test automation. This requires the study devoted to testingmay vary from 30% up to 70% of a project's of the scope of testingand the test plan .It is also needed to resources. Yet, testing is often an underexposedand know whether the test tool meets the test suite requirements for underestimated part of software development. Also testing are the particular product and version. The important factors that rather low compared with the importance of the subjectin real- also come into picture are reusability, reliability and cost. This life software projects. The course on testing techniques is is done to see if they can get the maximum benefit out the intended to be a steptowards bridging this gap.Testing product being made, bought or customized. The test tool techniques deals with a number of topics related to software should support: Scripting interface; Facility to give valid and testing. The aimis that the student will get insight into some of invalid input; Result comparison;To give the verdict.[15] the problems of software testing andthat he gets familiar with a couple of solutions to these problems. Emphasis ison III. TESTING TOOLS techniques for testing technical software systems, such as There are many testing tools available that are useful in several communication software,control systems and embedded places while testing software product. These tools canbe software. Established testing techniques as well as categorized as static testing and dynamic testing tools. newdevelopments will be presented. In the latter category there will be a strong focus onthe use of formal methods in the A. Static Testing Tools software testing process. It is impossible to give acomplete These tools test the software without executing it; rather, they view of all possible testing techniques and processes in a single are concerned with analyzing the code or documentation for course. Testing techniques is a reasonable compromise syntax checking, consistency, etc. Static testing can be manual between giving a completeoverview of the broad _end of or even automated with the use of static analysis tools. Static testing and dealing thoroughly with distinct testingtechniques analysis tools examine the source code of program and without being too super-cial. [13] highlight the statements with wrong syntax, undefined symbols This paper is organized in seven sections: In section or variables, use of uninitialized variables, and so on. They 1.Introduction; 2. Literature review; 3.Testing tools; 4. also check for flaws in the logic flow of the program. Static Advanced Software Testing Tools; 5. Testing Techniques; analysis tools are generally used by developers as part of the 6.Advanced Software Testing Techniques; 7. Conclusion. development and component testing process. The key aspect is that the code is not executed or run but the tool itself is II. LITERATURE REVIEW executed, and the source code we are interested in is the input Software testing is a very broad area, which involves many data to the tool. These tools are mostly used by other technical and non-technical areas, such asspecification, developers.Static analysis tools are an extension of compiler IJTRD | Jan – Feb 2019 Available [email protected] 29 International Journal of Trend in Research and Development, Volume 6(1), ISSN: 2394-9333 www.ijtrd.com technology – in fact some compilers do offer static analysis correctness of input and output, find and remove the errors, and features. It is worth checking what is available from existing control and summarize the test. Similarly, Bug Trapper is used compilers or development environments before looking at to perform white box testing. This tool traces the path of purchasing a more sophisticated static analysis tool. Other than execution and captures the bug along with the path of software code, static analysis can also be carried out on things execution and the different input values that resulted in the like, static analysis of requirements or static analysis of error[6].Dynamic analysis tools are „dynamic‟ because they websites. Static analysis tools for code can help the developers require the code to be in a running state. They are „analysis‟ to understand the structure of the code, and can also be used to rather than „testing‟ tools because they analyze what is enforce coding standards. [16] Static Testing happening „behind the scenes‟ that is in the code while the Techniques provide a powerful way to improve the quality and software is running (whether being executed with test cases or productivity of software development by assisting engineers to being used in operation). Eventually when our computer‟s recognize and fix their own defects early in the software response time gets slower and slower, but it get improved after development process. In this software is tested without re-booting, this may be because of the „memory leak‟, where executing the code by doing Review, Walk Through, the programs do not correctly release blocks of memory back Inspection or Analysis etc. Static Testing may be conducted to the operating system. Sooner or later the system will run out manually or through the use of various software testing tools. It of memory completely and stop. Hence, rebooting restores all starts early in the Software Development Life Cycle and so it is of the memory that was lost, so the performance of the system done during the Verification Process. Static testing is not is now restored to its normal state. These tools would typically magic and it should not be considered a replacement be used by developers in component testing and component for Dynamic Testing, but all software organizations should integration testing, e.g. when testing middleware, when testing consider using reviews in all major aspects of their work security or when looking for robustness defects. Dynamic including requirements, design, implementation, testing, and analysis for websites is to check whether each link does maintenance.Types of defects that are easier to find during actually link to something else. The tool does not know if you static testing are: deviations from standards, missing have linked to the correct page, but at least it can find dead requirements, design defects, non-maintainable code and links, which may be helpful. [16] In Dynamic Testing inconsistent interface specifications. [17] Static Testing is type Technique software is executed using a set of input values and of testing in which the code is not executed. It can be done its output is then examined and compared to what is expected. manually or by a set of tools. This type of testing checks the Dynamic execution is applied as a technique to detect defects code, requirement documents and design documents and puts and to determine quality attributes of the code.
Recommended publications
  • Webnet 2000 World Conference on the WWW and Internet Proceedings (San Antonio, Texas, October 30-November 4Th, 2000)

    Webnet 2000 World Conference on the WWW and Internet Proceedings (San Antonio, Texas, October 30-November 4Th, 2000)

    DOCUMENT RESUME ED 448 744 IR 020 507 AUTHOR Davies, Gordon, Ed.; Owen, Charles, Ed. TITLE WebNet 2000 World Conference on the WWW and Internet Proceedings (San Antonio, Texas, October 30-November 4th, 2000) . INSTITUTION Association for the Advancement of Computing in Education, Charlottesville, VA. ISBN ISBN-1-880094-40-1 PUB DATE 2000-11-00 NOTE 1005p.; For individual papers, see IR 020 508-527. For the 1999 conference, see IR 020 454. AVAILABLE FROM Association for the Advancement of Computing in Education (AACE), P.O. Box 3728, Norfolk, VA 23514-3728; Web site: http://www.aace.org. PUB TYPE Collected Works Proceedings (021) EDRS PRICE MF07/PC41 Plus Postage. DESCRIPTORS *Computer Uses in Education; Courseware; Distance Education; *Educational Technology; Electronic Libraries; Electronic Publishing; *Information Technology; Internet; Multimedia Instruction; Multimedia Materials; Postsecondary Education; *World Wide Web IDENTIFIERS Electronic Commerce; Technology Utilization; *Web Based Instruction ABSTRACT The 2000 WebNet conference addressed research, new developments, and experiences related to the Internet and World Wide Web. The 319 contributions of WebNet 2000 contained in this proceedings comprise the full and short papers accepted for presentation at the conference, as well as poster/demonstration abstracts. Major topics covered include: commercial, business, professional, and community applications; education applications; electronic publishing and digital libraries; ergonomic, interface, and cognitive issues; general Web tools and facilities; medical applications of the Web; \personal applications and environments;, societal issues, including legal, standards, and international issues; and Web technical facilities. (MES) Reproductions supplied by EDRS are the best that can be made from the original document. Web Net 2 World Conference Alb U.S.
  • Types of Software Testing

    Types of Software Testing

    Types of Software Testing We would be glad to have feedback from you. Drop us a line, whether it is a comment, a question, a work proposition or just a hello. You can use either the form below or the contact details on the rightt. Contact details [email protected] +91 811 386 5000 1 Software testing is the way of assessing a software product to distinguish contrasts between given information and expected result. Additionally, to evaluate the characteristic of a product. The testing process evaluates the quality of the software. You know what testing does. No need to explain further. But, are you aware of types of testing. It’s indeed a sea. But before we get to the types, let’s have a look at the standards that needs to be maintained. Standards of Testing The entire test should meet the user prerequisites. Exhaustive testing isn’t conceivable. As we require the ideal quantity of testing in view of the risk evaluation of the application. The entire test to be directed ought to be arranged before executing it. It follows 80/20 rule which expresses that 80% of defects originates from 20% of program parts. Start testing with little parts and extend it to broad components. Software testers know about the different sorts of Software Testing. In this article, we have incorporated majorly all types of software testing which testers, developers, and QA reams more often use in their everyday testing life. Let’s understand them!!! Black box Testing The black box testing is a category of strategy that disregards the interior component of the framework and spotlights on the output created against any input and performance of the system.
  • Smoke Testing What Is Smoke Testing?

    Smoke Testing What Is Smoke Testing?

    Smoke Testing What is Smoke Testing? Smoke testing is the initial testing process exercised to check whether the software under test is ready/stable for further testing. The term ‘Smoke Testing’ is came from the hardware testing, in the hardware testing initial pass is done to check if it did not catch the fire or smoked in the initial switch on.Prior to start Smoke testing few test cases need to created once to use for smoke testing. These test cases are executed prior to start actual testing to checkcritical functionalities of the program is working fine. This set of test cases written such a way that all functionality is verified but not in deep. The objective is not to perform exhaustive testing, the tester need check the navigation’s & adding simple things, tester needs to ask simple questions “Can tester able to access software application?”, “Does user navigates from one window to other?”, “Check that the GUI is responsive” etc. Here are graphical representation of Smoke testing & Sanity testing in software testing: Smoke Sanity Testing Diagram The test cases can be executed manually or automated; this depends upon the project requirements. In this types of testing mainly focus on the important functionality of application, tester do not care about detailed testing of each software component, this can be cover in the further testing of application. The Smoke testing is typically executed by testers after every build is received for checking the build is in testable condition. This type of testing is applicable in the Integration Testing, System Testing and Acceptance Testing levels.
  • Testing Web Services: a Survey Technical Report TR-10-01

    Testing Web Services: a Survey Technical Report TR-10-01

    Testing Web Services: A Survey Technical report TR-10-01 Mustafa Bozkurt, Mark Harman and Youssef Hassoun Centre for Research on Evolution, Search & Testing King’s College London Strand, London WC2R 2LS, UK mustafa.bozkurt,mark.harman,youssef.hassoun @kcl.ac.uk f g Abstract The Service-Oriented Computing (SOC) paradigm is allowing computer systems to interact with each other in new ways. According to the literature, SOC allows composition of distributed applications free from their platform and thus reduces the cost of such compositions and makes them easier and faster to develop. Currently web services are the most widely accepted service technology due to the level of autonomy and platform-independency they provide. However, web services also bring challenges. For example, testing web services at the client side is not as straightforward as testing traditional software due to the complex nature of web services and the absence of source code. This paper surveys the previous work undertaken on web service testing, showing the strengths and weaknesses of current web service testing strategies and identifying issues for future work. 1 Introduction This paper presents a survey of web service testing techniques. Web services is a rapidly growing concept that drives the Service-Oriented Computing (SOC) at present. Web services present important challenges to software testers. These challanges has led to much work on techniques for testing web services. The present paper seeks to provide a comprehensive survey of existing work. According to Papazoglou [110], SOC is a new computing paradigm that utilizes services as the lightweight constructs to support the development of rapid, low-cost and easy composition of distributed applications.
  • ® Voke Research

    ® Voke Research

    voke Research MARKET MOVER ARRAY™ REPORT: Service Virtualization By Theresa Lanowitz, Lisa Dronzek | September 26, 2018 Vendor Excerpt The completeclients publication at www.vokeinc.com is available to voke. Research ® Moving markets beyond the status quo! voke Research MARKET MOVER ARRAY™ REPORT: Service Virtualization By Theresa Lanowitz, Lisa Dronzek | September 26, 2018 ~ SUMMARY ~ TABLE OF CONTENTS Service virtualization is a powerful, Executive Overview 2 proven, and collaborative technology • Business Context 2 that enables software engineering • Technology Overview 2 teams to more accurately model • Market Status 3 the real-life behavior of software Market Mover Array Methodology 4 prior to production. By removing Market Mover Array Chart 6 the constraints and wait time for Market Mover Array Vendors 6 assets to be complete and available, • CA Technologies 9 service virtualization solutions deliver • IBM 11 on the promise of better business • Micro Focus 13 outcomes. Your teams can work • Parasoft 15 faster, release more • SmartBear 17 reliable software, • Tricentis 19 and delight the Related Research 21 business. • Market Specific 21 • Vendor Specific 21 • Book 21 © 2018 voke media, llc. All rights reserved. voke is a trademark of voke media, llc. and is registered in the U.S. All other trademarks are the property of their respective companies. Reproduction or distribution of this document except as expressly provided in writing by voke is strictly prohibited. Opinions reflect judgment at the time and are subject to change without notice. voke disclaims all warranties as to the accuracy, completeness or adequacy of information and shall have no liability for errors, omissions or inadequacies in the information contained or for interpretations thereof.
  • Parasoft Static Application Security Testing (SAST) for .Net - C/C++ - Java Platform

    Parasoft Static Application Security Testing (SAST) for .Net - C/C++ - Java Platform

    Parasoft Static Application Security Testing (SAST) for .Net - C/C++ - Java Platform Parasoft® dotTEST™ /Jtest (for Java) / C/C++test is an integrated Development Testing solution for automating a broad range of testing best practices proven to improve development team productivity and software quality. dotTEST / Java Test / C/C++ Test also seamlessly integrates with Parasoft SOAtest as an option, which enables end-to-end functional and load testing for complex distributed applications and transactions. Capabilities Overview STATIC ANALYSIS ● Broad support for languages and standards: Security | C/C++ | Java | .NET | FDA | Safety-critical ● Static analysis tool industry leader since 1994 ● Simple out-of-the-box integration into your SDLC ● Prevent and expose defects via multiple analysis techniques ● Find and fix issues rapidly, with minimal disruption ● Integrated with Parasoft's suite of development testing capabilities, including unit testing, code coverage analysis, and code review CODE COVERAGE ANALYSIS ● Track coverage during unit test execution and the data merge with coverage captured during functional and manual testing in Parasoft Development Testing Platform to measure true test coverage. ● Integrate with coverage data with static analysis violations, unit testing results, and other testing practices in Parasoft Development Testing Platform for a complete view of the risk associated with your application ● Achieve test traceability to understand the impact of change, focus testing activities based on risk, and meet compliance
  • Software Testing: Essential Phase of SDLC and a Comparative Study Of

    Software Testing: Essential Phase of SDLC and a Comparative Study Of

    International Journal of System and Software Engineering Volume 5 Issue 2, December 2017 ISSN.: 2321-6107 Software Testing: Essential Phase of SDLC and a Comparative Study of Software Testing Techniques Sushma Malik Assistant Professor, Institute of Innovation in Technology and Management, Janak Puri, New Delhi, India. Email: [email protected] Abstract: Software Development Life-Cycle (SDLC) follows In the software development process, the problem (Software) the different activities that are used in the development of a can be dividing in the following activities [3]: software product. SDLC is also called the software process ∑ Understanding the problem and it is the lifeline of any Software Development Model. ∑ Decide a plan for the solution Software Processes decide the survival of a particular software development model in the market as well as in ∑ Coding for the designed solution software organization and Software testing is a process of ∑ Testing the definite program finding software bugs while executing a program so that we get the zero defect software. The main objective of software These activities may be very complex for large systems. So, testing is to evaluating the competence and usability of a each of the activity has to be broken into smaller sub-activities software. Software testing is an important part of the SDLC or steps. These steps are then handled effectively to produce a because through software testing getting the quality of the software project or system. The basic steps involved in software software. Lots of advancements have been done through project development are: various verification techniques, but still we need software to 1) Requirement Analysis and Specification: The goal of be fully tested before handed to the customer.
  • Making the Case for a Sound Type System in Javascript: Here's JS++

    Making the Case for a Sound Type System in Javascript: Here's JS++

    SDT328 cover_Layout 1 7/22/16 2:25 PM Page 1 A BZ Media Publication Making the case for Don’t let testing a sound type system crimp your agility. in JavaScript: Here’s JS++ In fact, put it first! AUGUST 2016 • ISSUE NO. 328 • $9.95 • www.sdtimes.com SDT328 Full Page Ads_Layout 1 7/21/16 3:07 PM Page 2 Other Mobile Printers Just Don’t Cut It. Improve your bottom line with the Epson® Mobilink™ P80 Plus. It’s the only 3" mobile receipt printer with a built-in auto cutter, to get customers in and out faster. The compact P80 Plus is just the latest innovation from the Mobilink line of printers that are a cut above ordinary receipt printers. Mobilink printers feature best-in-class battery life, the fl exibility to be used for stationary or mobile POS applications, rock-solid durability and seamless integration with mobile devices. epson.com/Mobilink Mobilink Printers Mobilink P80 Plus P80 P60II P20 3" receipt 2” receipt/label 2” receipt Specifi cations subject to change without notice. EPSON is a registered trademark, EPSON Exceed Your Vision is a registered logomark of Seiko Epson Corporation. All other product and brand names are trademarks and/or registered trademarks of their respective companies. NFC Wireless Epson disclaims any and all rights in these marks. Copyright 2016 Epson America, Inc. SDT328 Full Page Ads_Layout 1 7/21/16 3:08 PM Page 3 Data entry errors. Consolidation headaches. Quality data shouldn’t be a pain. Try a little Vitamin Q. We supply Vitamin Q – “quality” – in data quality management solutions that profile, clean, enrich, and match your customer data – and keep it healthy over time.
  • Security Testing TDDC90 – Software Security

    Security Testing TDDC90 – Software Security

    Security Testing TDDC90 – Software Security Ulf Kargén Department of Computer and Information Science (IDA) Division for Database and Information Techniques (ADIT) Security testing vs “regular” testing ▪ “Regular” testing aims to ensure that the program meets customer requirements in terms of features and functionality. ▪ Tests “normal” use cases Test with regards to common expected usage patterns. ▪ Security testing aims to ensure that program fulfills security requirements. ▪ Often non-functional. ▪ More interested in misuse cases Attackers taking advantage of “weird” corner cases. 2 Functional vs non-functional security requirements ▪ Functional requirements – What shall the software do? ▪ Non-functional requirements – How should it be done? ▪ Regular functional requirement example (Webmail system): It should be possible to use HTML formatting in e-mails ▪ Functional security requirement example: The system should check upon user registration that passwords are at least 8 characters long ▪ Non-functional security requirement example: All user input must be sanitized before being used in database queries How would you write a unit test for this? 3 Common security testing approaches Often difficult to craft e.g. unit tests from non-functional requirements Two common approaches: ▪ Test for known vulnerability types ▪ Attempt directed or random search of program state space to uncover the “weird corner cases” In today’s lecture: ▪ Penetration testing (briefly) ▪ Fuzz testing or “fuzzing” ▪ Concolic testing 4 Penetration testing ▪ Manually try to “break” software ▪ Relies on human intuition and experience. ▪ Typically involves looking for known common problems. ▪ Can uncover problems that are impossible or difficult to find using automated methods ▪ …but results completely dependent on skill of tester! 5 Fuzz testing Idea: Send semi-valid input to a program and observe its behavior.
  • Accelerate Software Innovation Through Continuous Quality

    Accelerate Software Innovation Through Continuous Quality

    Accelerate Software Innovation Through Continuous Quality 1 Software quality is recognized as the #1 issue IT executives are trying to mitigate. Enterprise organizations strive to accelerate the delivery of a compelling user experience to their customers in order to drive revenue. Software quality is recognized as the #1 issue IT executives are trying to mitigate. QA teams know they have issues and are actively looking for solutions to save time, increase quality, improve security, and more. The most notable difficulties are in identifying the right areas to test, the availability of flexible and reliable test environments and test data, and the realization of benefits from automation. You may be facing many challenges with delivering software to meet the high expectations for quality, cost, and schedule driven by the business. An effective software testing strategy can address these issues. If you’re looking to improve your software quality while achieving your business goals, Parasoft can help. With over 30 years of making testing easier for our customers, we have the innovation you need and the experience you trust. Our extensive continuous quality suite spans every testing need and enables you to reach new heights. 3 QUALITY-FIRST APPROACH You can’t test quality into an application at the end of the software development life cycle (SDLC). You need to ensure that your software development process and practices put a priority on quality- driven development and integrate a comprehensive testing strategy to verify that the application’s functionality meets the requirements. Shift testing left to the start of your development process to bring quality to the forefront.
  • Model-Based Security Testing

    Model-Based Security Testing

    Model-Based Security Testing Ina Schieferdecker Juergen Grossmann Fraunhofer FOKUS Fraunhofer FOKUS Berlin, Germany Berlin, Germany Freie Universitaet Berlin [email protected] Berlin, Germany [email protected] Martin Schneider Fraunhofer FOKUS Berlin, Germany [email protected] Security testing aims at validating software system requirements related to security properties like confidentiality, integrity, authentication, authorization, availability, and non-repudiation. Although security testing techniques are available for many years, there has been little approaches that allow for specification of test cases at a higher level of abstraction, for enabling guidance on test identification and specification as well as for automated test generation. Model-based security testing (MBST) is a relatively new field and especially dedicated to the sys- tematic and efficient specification and documentation of security test objectives, security test cases and test suites, as well as to their automated or semi-automated generation. In particular, the com- bination of security modelling and test generation approaches is still a challenge in research and of high interest for industrial applications. MBST includes e.g. security functional testing, model-based fuzzing, risk- and threat-oriented testing, and the usage of security test patterns. This paper provides a survey on MBST techniques and the related models as well as samples of new methods and tools that are under development in the European ITEA2-project DIAMONDS. 1 Introduction The times of rather static communication in strictly controlled, closed networks for limited purposes are over, while the adoption of the Internet and other communication technologies in almost all domes- tic, economic and social sectors with new approaches for rather dynamic and open networked environ- ments overwhelmingly progresses.
  • Guidelines on Minimum Standards for Developer Verification of Software

    Guidelines on Minimum Standards for Developer Verification of Software

    Guidelines on Minimum Standards for Developer Verification of Software Paul E. Black Barbara Guttman Vadim Okun Software and Systems Division Information Technology Laboratory July 2021 Abstract Executive Order (EO) 14028, Improving the Nation’s Cybersecurity, 12 May 2021, di- rects the National Institute of Standards and Technology (NIST) to recommend minimum standards for software testing within 60 days. This document describes eleven recommen- dations for software verification techniques as well as providing supplemental information about the techniques and references for further information. It recommends the following techniques: • Threat modeling to look for design-level security issues • Automated testing for consistency and to minimize human effort • Static code scanning to look for top bugs • Heuristic tools to look for possible hardcoded secrets • Use of built-in checks and protections • “Black box” test cases • Code-based structural test cases • Historical test cases • Fuzzing • Web app scanners, if applicable • Address included code (libraries, packages, services) The document does not address the totality of software verification, but instead recom- mends techniques that are broadly applicable and form the minimum standards. The document was developed by NIST in consultation with the National Security Agency. Additionally, we received input from numerous outside organizations through papers sub- mitted to a NIST workshop on the Executive Order held in early June, 2021 and discussion at the workshop as well as follow up with several of the submitters. Keywords software assurance; verification; testing; static analysis; fuzzing; code review; software security. Disclaimer Any mention of commercial products or reference to commercial organizations is for infor- mation only; it does not imply recommendation or endorsement by NIST, nor is it intended to imply that the products mentioned are necessarily the best available for the purpose.