DOCSLIB.ORG

Software Testing and Debugging

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Software Testing and Debugging TheServerSide.com SearchSoftwareQuality.com TheServerSide Java Symposium TechTarget Application Development Media TheServerSide.NET SearchWebServices.com The Ajax Experience Ajaxian.com SearchVB.com E-Guide Software Testing and Debugging Software Testing and Debugging provides an overview of how to detect and test code performance and security vulnerabilities early in the development cycle; best practices for Web application source code analysis; how to identify and resolve race conditions and other concurrency defects; understand the complex interleavings that exist within multi-threaded applications; implement - ing fundamental secure development practices throughout the SDLC and more. Sponsored By: Software Testing and Debugging Table of Contents E-Guide Software Testing and Debugging Table of Contents: Eight reasons to do source code analysis on your Web application Static code analyzer tests for dangerous race conditions Application security shouldn’t involve duct tape, Band-Aids or bubble gum Resources from Coverity Sponsored by: Page 2 of 15 Software Testing and Debugging Eight reasons to do source code analysis on your Web applications Eight reasons to do source code analysis on your Web application By Kevin Beaver Given the visibility and vulnerabilities that make up many Web applications, I’m surprised to see that the underlying source code isn’t being analyzed. Be it ignorance, lack of budget or anything in between, there’s a lot that can come from performing this exercise in the name of reducing business risks. Here are eight reasons for placing a higher priority analyzing the source code of your Web applications: 1. It’s easy to do Source code analysis sounds very technical and hands-on. Who outside of the techiest of all developers and IT byteheads wants to get involved with something like that? Actually, this difficulty factor is a myth. Once you get analysis tools installed (which happens to be the hardest part I’ve experienced), it’s simply a matter of pointing them to the source files and clicking Go. The tools I’ve used are mature and chock-full of handy features that most anyone can use without expending too much brain power. 2. High benefit and (relatively) low cost I’ve yet to perform a source code analysis where the developers haven’t come back and said, “Oh, I didn’t think of that.” In many cases, the vulnerabilities that source code analysis tools have found wouldn’t have been discovered by developers in a million years but could be exploited at any minute by an attacker. Finding just one vulnerability easily justifies the few thousand dollars you’ll spend on any tools you’ll buy. 3. Web applications are a prime target Think about where Web applications are located. They’re accessible to any and all. Both internal users and external attackers can connect and do bad things—oftentimes even without a valid user account. Contributing to the prob - lem is the fact that most attacks will go unnoticed even with logging enabled and basic log auditing taking place. There’s just too much noise to keep up with. Finally, many Web applications are the entry point to all that makes up the business. There’s so much accessibility and so much to lose. 4. Security is as important as functionality and the user experience It’s funny how so many people—testers, QA engineers, product managers and even upper management—are often involved in making sure software is just right in the eyes of the customer. As long as the software does what it’s sup - posed do, that’s all that really matters. But what about the security flaws that could impact the user on a grander scale? The priorities are shifting slowly, but the “requirement” of secure code is still not where it needs to be. 5. Traditional security testing tools go only so far I’m a firm believer that you’ve got to have good tools to find good vulnerabilities. I’ve also experienced how imper - fect Web application vulnerability scanning tools can be. You often have to manually validate what the tools find and then some. But manual analysis is not going to uncover everything either. That leaves the third and final option— Sponsored by: Page 3 of 15 Software Testing and Debugging Eight reasons to do source code analysis on your Web applications auditing checklists. But we all know how little value those provide. You’ve got to look at the entire attack surface of your Web applications from all possible angles. That includes the source code. 6. Enhanced development processes A nice side benefit of source code analysis is that it will often uncover weaknesses in the software development lifecycle (SDLC) and broader problems with business processes that would otherwise go unnoticed. Plus, security integration earlier on will save time, effort and money for everyone involved. 7. Competitive differentiation A lot of people are talking about source code analysis. I don’t buy into vendor hype, but I do listen when end users start talking about issues such as this. Apparently the need is growing. A lot of security assessment work is the result of a third party -- be it a customer, auditor or business partner—asking for an independent view of software vulnerabilities. When the time comes for someone to ask, “How do I know your software is secure?” it will pay to have a good response and perhaps a source code analysis report to share with him. 8. Compliance as a side effect Be it meeting service-level agreements (SLAs), internal policies or the mish-mosh of industry and government regu - lations we’re all up against, having higher-quality source code can help businesses meet their compliance and legal requirements. In fact, fixing the vulnerabilities in your Web applications could be the one thing that makes the business compliant or helps it look better on its next audit. Arguably many businesses could benefit from certain financial gains as the result of growing the customer base, building trust and loyalty, and putting out more and more software. Those benefits come when they root out Web security problems where they often begin—in the source code—and create more secure and higher-quality software. Consider giving source code analysis a whirl. Most tool vendors provide some sort of free trial of their products. It won’t hurt to at least try it out to see what’s uncovered. It may end up being the best thing you do to improve software quality and minimize security risks for some time to come. About the author: Kevin Beaver is an independent information security consultant, speaker, and expert witness with Atlanta-based Principle Logic, LLC. He has more than 19 years of experience in IT and specializes in perform - ing information security assessments revolving around IT compliance. Kevin has authored/co-authored six books on information security including Hacking For Dummies and Hacking Wireless Networks For Dummies (Wiley) as well as The Practical Guide to HIPAA Privacy and Security Compliance (Auerbach). He’s also the creator of the Security On Wheels series of audiobooks. Sponsored by: Page 4 of 15 UNDETECTED DEFECTS LURKING IN YOUR CODE CAN PROVE DISASTROUS FOR YOUR BUSINESS. Hidden bugs in your valuable source code can have serious consequences for your software—countless patches, drops in customer satisfaction, product recalls or worse. You need to know all your code is clean. Coverity offers advanced source code analysis products for the detection of hazardous defects and security vulnerabilities. Catastrophic errors are identified immediately as code is written. Assuring the highest possible code quality—no matter how complex your code base. This allows your developers to spend less time searching for bugs and more time adding value to your product. Your code is either coverity clean—or it’s not. Ceratitis Capitata, or Medfly Unchecked, crop damage estimated at $821 million. Electron Micrograph, 80X © 2008 Coverity, Inc. All rights reserved. © 2008 Coverity, Software Testing and Debugging Static code analyzer tests for dangerous race conditions Static code analyzer tests for dangerous race conditions By Colleen Frye Testing for race condition defects in code is like looking for the proverbial needle in the haystack, according to Ben Chelf, chief technology officer of Coverity Inc. “You might see something strange, but you may not see the same behavior twice in row, and you don’t have complete control of the system, even in the test lab.” Now, with new concurrency defect detection capabilities in the Coverity Prevent SQS static code analyzer, develop - ers will be about to identify race conditions and other concurrency defects earlier in the development cycle, before it gets to test, according to the company. With the advent of multi-core hardware, developers are being pushed to develop multi-threaded applications. In multi-threaded applications, race conditions occur when two threads are trying to access the same data at the same time, which can corrupt the data. “Developers need to develop with multiple threads to take advantage of multi-cores,” Chelf said. “With single-core, it was less critical for developers to think multithreaded development. With single-threaded programming, the developer has complete control over what happens—they can see every condition and determine what will happen when you run a program.” With multithread there’s a combination of what’s happening in code, but it’s also dependent on the platform operating system where the code is running. The operating system starts making decisions for you; it schedules the thread. If the program is not designed well, it can make the program behave differently depending on what the operating system does with the threads, Chelf said. So while testers can exercise the system to look for race conditions, “it’s almost impossible to test for all the paths in the code and interleaving threads, and developers don’t have control over how threads are to be scheduled,” Chelf continued.
Load more

Recommended publications
  • Testing E-Commerce Systems: a Practical Guide - Wing Lam
    Testing E-Commerce Systems: A Practical Guide - Wing Lam As e-customers (whether business or consumer), we are unlikely to have confidence in a Web site that suffers frequent downtime, hangs in the middle of a transaction, or has a poor sense of usability. Testing, therefore, has a crucial role in the overall development process. Given unlimited time and resources, you could test a system to exhaustion. However, most projects operate within fixed budgets and time scales, so project managers need a systematic and cost- effective approach to testing that maximizes test confidence. This article provides a quick and practical introduction to testing medium- to large-scale transactional e-commerce systems based on project experiences developing tailored solutions for B2C Web retailing and B2B procurement. Typical of most e-commerce systems, the application architecture includes front-end content delivery and management systems, and back-end transaction processing and legacy integration. Aimed primarily at project and test managers, this article explains how to · establish a systematic test process, and · test e-commerce systems. The Test Process You would normally expect to spend between 25 to 40 percent of total project effort on testing and validation activities. Most seasoned project managers would agree that test planning needs to be carried out early in the project lifecycle. This will ensure the time needed for test preparation (establishing a test environment, finding test personnel, writing test scripts, and so on) before any testing can actually start. The different kinds of testing (unit, system, functional, black-box, and others) are well documented in the literature.
    [Show full text]
  • Interpreting Reliability and Availability Requirements for Network-Centric Systems What MCOTEA Does
    Marine Corps Operational Test and Evaluation Activity Interpreting Reliability and Availability Requirements for Network-Centric Systems What MCOTEA Does Planning Testing Reporting Expeditionary, C4ISR & Plan Naval, and IT/Business Amphibious Systems Systems Test Evaluation Plans Evaluation Reports Assessment Plans Assessment Reports Test Plans Test Data Reports Observation Plans Observation Reports Combat Ground Service Combat Support Initial Operational Test Systems Systems Follow-on Operational Test Multi-service Test Quick Reaction Test Test Observations 2 Purpose • To engage test community in a discussion about methods in testing and evaluating RAM for software-intensive systems 3 Software-intensive systems – U.S. military one of the largest users of information technology and software in the world [1] – Dependence on these types of systems is increasing – Software failures have had disastrous consequences Therefore, software must be highly reliable and available to support mission success 4 Interpreting Requirements Excerpts from capabilities documents for software intensive systems: Availability Reliability “The system is capable of achieving “Average duration of 716 hours a threshold operational without experiencing an availability of 95% with an operational mission fault” objective of 98%” “Mission duration of 24 hours” “Operationally Available in its intended operating environment “Completion of its mission in its with at least a 0.90 probability” intended operating environment with at least a 0.90 probability” 5 Defining Reliability & Availability What do we mean reliability and availability for software intensive systems? – One consideration: unlike traditional hardware systems, a highly reliable and maintainable system will not necessarily be highly available - Highly recoverable systems can be less available - A system that restarts quickly after failures can be highly available, but not necessarily reliable - Risk in inflating availability and underestimating reliability if traditional equations are used.
    [Show full text]
  • E2 Studio Integrated Development Environment User's Manual: Getting
    User’s Manual e2 studio Integrated Development Environment User’s Manual: Getting Started Guide Target Device RX, RL78, RH850 and RZ Family Rev.4.00 May 2016 www.renesas.com Notice 1. Descriptions of circuits, software and other related information in this document are provided only to illustrate the operation of semiconductor products and application examples. You are fully responsible for the incorporation of these circuits, software, and information in the design of your equipment. Renesas Electronics assumes no responsibility for any losses incurred by you or third parties arising from the use of these circuits, software, or information. 2. Renesas Electronics has used reasonable care in preparing the information included in this document, but Renesas Electronics does not warrant that such information is error free. Renesas Electronics assumes no liability whatsoever for any damages incurred by you resulting from errors in or omissions from the information included herein. 3. Renesas Electronics does not assume any liability for infringement of patents, copyrights, or other intellectual property rights of third parties by or arising from the use of Renesas Electronics products or technical information described in this document. No license, express, implied or otherwise, is granted hereby under any patents, copyrights or other intellectual property rights of Renesas Electronics or others. 4. You should not alter, modify, copy, or otherwise misappropriate any Renesas Electronics product, whether in whole or in part. Renesas Electronics assumes no responsibility for any losses incurred by you or third parties arising from such alteration, modification, copy or otherwise misappropriation of Renesas Electronics product. 5. Renesas Electronics products are classified according to the following two quality grades: “Standard” and “High Quality”.
    [Show full text]
  • QA and Testing Have Become More of a Concurrent Process
    Special Feature: SW Testing “QA and testing have become more of a concurrent process rather than an end of lifecycle process” Manish Tandon, VP & Head– Independent Validation and Testing Solutions, Infosys Interview by: Anil Chopra, PCQuest n an exclusive interview with the PCQuest Editor, Manish talks become a software tester? about the global trends in software testing, skillsets required Software testing has evolved into a very special function and Ito enter the field, how technology has evolved in this area to anybody wanting to enter the domain requires two special provide higher ROI, and much more. An IIT and IIM graduate, skillsets. One is deep functional or domain skills to understand Manish is responsible for formulating and executing the business the functionality. Plus, you need specialized technical skills strategy for Independent Validation and Testing Solutions practice as well because you want to do automation, middleware, and at Infosys. He mentors the unit specifically in meeting business performance testing. It’s a unique combination, and we focus goals and targets. In addition, he manages critical relationships very strongly on both these dimensions. A good software tester with client executives, industry analysts, deal consultants, and should always have an eye for detail. So people who have a slightly anchors the training and development of key personnel. Provided critical eye and are willing to dig deeper always make much better here are his expert comments on the subject. testers than people who want to fly at 30k feet. Q> What are some of the global trends in the software Q> You mentioned that software testing is now required testing business? How do you see the market moving? for front-end applications.
    [Show full text]
  • SAS® Debugging 101 Kirk Paul Lafler, Software Intelligence Corporation, Spring Valley, California
    PharmaSUG 2017 - Paper TT03 SAS® Debugging 101 Kirk Paul Lafler, Software Intelligence Corporation, Spring Valley, California Abstract SAS® users are almost always surprised to discover their programs contain bugs. In fact, when asked users will emphatically stand by their programs and logic by saying they are bug free. But, the vast number of experiences along with the realities of writing code says otherwise. Bugs in software can appear anywhere; whether accidentally built into the software by developers, or introduced by programmers when writing code. No matter where the origins of bugs occur, the one thing that all SAS users know is that debugging SAS program errors and warnings can be a daunting, and humbling, task. This presentation explores the world of SAS bugs, providing essential information about the types of bugs, how bugs are created, the symptoms of bugs, and how to locate bugs. Attendees learn how to apply effective techniques to better understand, identify, and repair bugs and enable program code to work as intended. Introduction From the very beginning of computing history, program code, or the instructions used to tell a computer how and when to do something, has been plagued with errors, or bugs. Even if your own program code is determined to be error-free, there is a high probability that the operating system and/or software being used, such as the SAS software, have embedded program errors in them. As a result, program code should be written to assume the unexpected and, as much as possible, be able to handle unforeseen events using acceptable strategies and programming methodologies.
    [Show full text]
  • Agile Playbook V2.1—What’S New?
    AGILE P L AY B O OK TABLE OF CONTENTS INTRODUCTION ..........................................................................................................4 Who should use this playbook? ................................................................................6 How should you use this playbook? .........................................................................6 Agile Playbook v2.1—What’s new? ...........................................................................6 How and where can you contribute to this playbook?.............................................7 MEET YOUR GUIDES ...................................................................................................8 AN AGILE DELIVERY MODEL ....................................................................................10 GETTING STARTED.....................................................................................................12 THE PLAYS ...................................................................................................................14 Delivery ......................................................................................................................15 Play: Start with Scrum ...........................................................................................15 Play: Seeing success but need more fexibility? Move on to Scrumban ............17 Play: If you are ready to kick of the training wheels, try Kanban .......................18 Value ......................................................................................................................19
    [Show full text]
  • Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK
    1 From Start-ups to Scale-ups: Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK Abstract—This paper1 describes some of the challenges and research questions that target the most productive intersection opportunities when deploying static and dynamic analysis at we have yet witnessed: that between exciting, intellectually scale, drawing on the authors’ experience with the Infer and challenging science, and real-world deployment impact. Sapienz Technologies at Facebook, each of which started life as a research-led start-up that was subsequently deployed at scale, Many industrialists have perhaps tended to regard it unlikely impacting billions of people worldwide. that much academic work will prove relevant to their most The paper identifies open problems that have yet to receive pressing industrial concerns. On the other hand, it is not significant attention from the scientific community, yet which uncommon for academic and scientific researchers to believe have potential for profound real world impact, formulating these that most of the problems faced by industrialists are either as research questions that, we believe, are ripe for exploration and that would make excellent topics for research projects. boring, tedious or scientifically uninteresting. This sociological phenomenon has led to a great deal of miscommunication between the academic and industrial sectors. I. INTRODUCTION We hope that we can make a small contribution by focusing on the intersection of challenging and interesting scientific How do we transition research on static and dynamic problems with pressing industrial deployment needs. Our aim analysis techniques from the testing and verification research is to move the debate beyond relatively unhelpful observations communities to industrial practice? Many have asked this we have typically encountered in, for example, conference question, and others related to it.
    [Show full text]
  • Usage of Prototyping in Software Testing
    Multi-Knowledge Electronic Comprehensive Journal For Education And Science Publications (MECSJ) ISSUE (14), Nov (2018) www.mescj.com USAGE OF PROTOTYPING IN SOFTWARE TESTING 1st Khansaa Azeez Obayes Al-Husseini Babylon Technical Insitute , Al-Furat Al-Awsat Technical University,51015 Babylon,Iraq. 1st [email protected] 2nd Ali Hamzah Obaid Babylon Technical Insitute , Al-Furat Al-Awsat Technical University,51015 Babylon,Iraq. 2nd [email protected] Abstract: Prototyping process is an important part of software development. This article describes usage of prototyping using Question – and – Answer memory and visual prototype diesign to realize Prototyping software development model. It also includes review of different models of software lifecycle with comparison them with Prototyping model. Key word: Question – and – Answer , Prototype, Software Development, RAD model. 1. Introduction One of the most important parts of software development is project design. Software project designing as a process of project creation can be divided in two large parts (very conditional): design of the functionality and design of user interface. To design the functionality, tools such as UML and IDEF0 are used, which have already become industry standards for software development. In the design of the graphical user interface there are no established standards, there are separate recommendations, techniques, design features, traditions, operating conditions for software, etc. At the same time, an important, but not always properly performed, part of this process is prototyping, i.e. the creation of a prototype or prototype of a future system. Prototypes can be different: paper, presentation, imitation, etc., up to exact correspondence to the future program. Most of the modern integrated development environments for software (IDE) allows to create something similar to prototypes, but it is connected with specific knowledge of IDE and programming language.
    [Show full text]
  • Lecturer: Ting Wang (王挺)
    Lecturer: Ting Wang (王挺) 利物浦大学计算机博士 清华大学计算机博士后 电子信息技术高级工程师 上海外国语大学网络与新媒体副教授 浙江清华长三角研究院海纳认知与智能研究中心主任 New Media Product DevelopmentDesign and Dr. Ting WANG School of Journalism and Communication Haina Cognition and Intelligence Research Center Shanghai International Studies University Yangtze Delta Region Institute of Tsinghua University, Zhejiang An overview to Part 01 software testing Testing in the software development Project Manager (Requirement Analyzer) System Analysis Release Structure Designer System Analysis Quality Assurance Engineer Requirement Analyzer Structure Design Testing Structure Design Programmer Structure Designer V-model Coding Coding Programmer Waterfall model Testing Quality Assurance Engineer Release Project Manager What is software testing? Software Testing Definition according to ANSI/IEEE 1059 standard -- A process of analyzing a software item to detect the differences between existing and required conditions (i.e., defects) and to evaluate the features of the software item. https://www.softwaretestingmaterial.com/software-testing/ “Testing shows the presence, not the absence of bugs.” —Edsger W. Dijkstra Edsger W. Dijkstra (May 11, 1930 - Aug 06, 2002) was a Dutch systems scientist, programmer, software engineer, science essayist, and pioneer in computing science. Bug Fault (also called “defect” or “bug”) is an erroneous hardware or software element of a system that can cause the system to fail. Reasons why software has bugs human mistakes in software design and coding. The longer a software bug exists throughout the product life-cycle, the more it costs. In 2002, software bugs cost the United States economy approximately $59.5 billion. In 2016, that number jumped to $1.1 trillion. https://dzone.com/articles/api-testing-best-practices 20 reasons for software bugs Reasons in development Reasons in testing #1) Miscommunication or No Communication #11) Not having a proper test setup (test environment) for testing all requirements.
    [Show full text]
  • Automated Testing of Firmware Installation and Update Scenarios for Peripheral Devices
    DEGREE PROJECT IN COMPUTER SCIENCE AND ENGINEERING, SECOND CYCLE, 30 CREDITS STOCKHOLM, SWEDEN 2019 Automated testing of firmware installation and update scenarios for peripheral devices DAG REUTERSKIÖLD KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE Automated testing of firmware installation and update scenarios for peripheral devices DAG REUTERSKIÖLD Master in Computer Science Date: August 12, 2019 Supervisor: Hamid Faragardi Examiner: Elena Troubitsyna School of Electrical Engineering and Computer Science Host company: Tobii AB Swedish title: Automatisering av enhetsinstallation, uppdatering och testning med hjälp av virtuella maskiner iii Abstract This research presents an approach to transition from manual to automated testing of hardware specific firmware. The manual approach for firmware test- ing can be repetitive and time consuming. A significant proportion of the time is spent on cleaning and re-installing operating systems so that old firmware does not interfere with the newer firmware that is being tested. The approach in this research utilizes virtual machines and presents an automation framework. One component of the automation framework is an application to imitate con- nected peripheral devices to bypass hardware dependencies of firmware in- stallers. The framework also consists of automation and pipeline scripts with the objective to execute firmware installers and detect errors and abnormalities in the installation and updating processes. The framework can run on locally hosted virtual machines, but is most applicable using cloud hosted virtual ma- chines, where it is part of a continuous integration that builds, downloads, installs, updates and tests new firmware versions, in a completely automated manner. The framework is evaluated by measuring and comparing execution times with manually conducted installation and updating tests, and the result shows that the framework complete tests much faster than the manual approach.
    [Show full text]
  • Software Testing Revealed Training Book Second Edition by International Software Test Institute™
    SOFTWARE TESTING REVEALED TRAINING BOOK SECOND EDITION BY INTERNATIONAL SOFTWARE TEST INSTITUTE™ www.test-institute.org © COPYRIGHT INTERNATIONAL SOFTWARE TEST INSTITUTE™ Dedication To all of the International Software Test Institute™ students, thank you for inspiring us, keeping us focused, and making sure we do our best to help you grow in your career with your skills and knowhow. Without you, your engagement and your loyal support, International Software Test Institute™ could not come where it is today. TABLE OF CONTENTS CLICKABLE WELCOME ........................................................................................................................6 ABOUT INTERNATIONAL SOFTWARE TEST INSTITUTE™ .....................................................7 Introduction To Software Testing ............................................................................8 What is Software Quality Assurance? ...................................................................12 What Is Software Testing? .......................................................................................18 Fundamentals of Software Testing ........................................................................21 Software Testing Roles and Responsibilities .......................................................30 Software Testing Methods ......................................................................................36 Software Testing Levels ..........................................................................................38 Software Testing
    [Show full text]
  • Empirical Evaluation of the Effectiveness and Reliability of Software Testing Adequacy Criteria and Reference Test Systems
    Empirical Evaluation of the Effectiveness and Reliability of Software Testing Adequacy Criteria and Reference Test Systems Mark Jason Hadley PhD University of York Department of Computer Science September 2013 2 Abstract This PhD Thesis reports the results of experiments conducted to investigate the effectiveness and reliability of ‘adequacy criteria’ - criteria used by testers to determine when to stop testing. The research reported here is concerned with the empirical determination of the effectiveness and reliability of both tests sets that satisfy major general structural code coverage criteria and test sets crafted by experts for testing specific applications. We use automated test data generation and subset extraction techniques to generate multiple tests sets satisfying widely used coverage criteria (statement, branch and MC/DC coverage). The results show that confidence in the reliability of such criteria is misplaced. We also consider the fault-finding capabilities of three test suites created by the international community to serve to assure implementations of the Data Encryption Standard (a block cipher). We do this by means of mutation analysis. The results show that not all sets are mutation adequate but the test suites are generally highly effective. The block cipher implementations are also seen to be highly ‘testable’ (i.e. they do not mask faults). 3 Contents Abstract ............................................................................................................................ 3 Table of Tables ...............................................................................................................
    [Show full text]