DOCSLIB.ORG

Devops; a Testers Best Friend

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

DevOps: A Testers Best Friend CHICAGO April 18th — April 22th Renaissance Hotel 1 West Wacker Drive Chicago IL 60601 Speaker(s): Ken Robertson & Thomas Janik When: April 21, 2016 Company: American Family Insurance Time: 3:30 pm - 4:30 pm DEVOPS; A TESTER’S BEST FRIEND Press ESC to continue DEVOPS; A TESTER’S BEST FRIEND TOM JANIK – AMERICAN FAMILY INSURANCE DEVOPS MANAGER – RELEASE MANAGEMENT KEN ROBERTSON – AMERICAN FAMILY INSURANCE SQA TEST MANAGER – SPECIALIZED TESTING OBJECTIVE OF SESSION: WALK THROUGH THE AMERICAN FAMILY JOURNEY TO BECOMING A COMPANY THAT OPERATES IN A CULTURE BASED IN DEVOPS IN THE BEGINNING: RELEASE FLOW DEVELOPMENT LEG DEV INT SIT LEG DEV INT QA RELEASE LEG DEV INT QA PERF PROD Deploys done by development teams, could happen at any time No stability requirements, no triage of deployment / environment issues 4 year development cycle using Amgile SDLC (selective Agile) Development “Leg” (flow), SIT “Leg”, 2 month effort in Release “Leg” IN THE BEGINNING: SIT TESTING LOCAL/DEV Development and Unit Testing DEV/INT Functional Testing QA Test Data Creation TDC SIT TDC Elevation 1 Elevation 2 Elevation … n Manual testing Testing led by development teams, lacked integration coordination 2 year SIT cycle with multiple code elevations No formalized automation or performance testing No shared test data, test scenarios, or documented financial testing Limited communication between development and testing – throw it over the wall mentality FORMING: THE DEVOPS REVOLUTION DEVELOPMENT LEG 1 DEV INT QA DEVELOPMENT LEG 2 DEV INT QA DEVELOPMENT LEG 3 DEV INT QA RELEASE LEG DEV INT QA PERF PROD D: Release Coordination, Automation, Performance, and Manual Testing Teams O: Change Management, Build and Deploy, Software Configuration Management, Infrastructure Coordination Teams Specific deployment windows, DENT environments, Floating QA environment Branching and merging strategies begin, targeted deploys to PERF (testing left) 3 parallel development (“Legs”) streams, Monthly maintenance release “Leg” Press ESC to continue FORMING: SIT TESTING CYCLES LOCAL/DEV Development and Unit Testing DEV/INT/DENT Functional Testing INT AIT (Application Integration Testing) Phase 1 Phase 2 Phase 3 QA HIST Test Data Creation QA SIT (System Integration Testing) Cycle 1 Cycle 2 Cycle n... Introduction of DENT and QA HIST tiers Point to point integration testing coordinated by development teams Application integration coordination being introduced 6 month SIT duration – 3 months financial infrastructure testing and 3 months for financials Documented financial testing in collaboration with controllers ACTIVITY: DEVOPS WHAT’S IMPORTANT TO YOU? Using the index card in front of you write down 3 things that are most important to you about DevOps. (1:00 Minute) Gather in groups of #-# at one of the posters on the wall. (1:00 Minute) Come up with something common between everyone in the group and write that on top of the poster as your group name. (1:00 Minute) Come up with one common thing from everyone’s index cards and draw (no words) that one common thing on the rest on the space on the poster. (1:00 Minute) STORMING: A CULTURE CHANGE TO DEVOPS DevOps is the practice of operations and development engineers participating together in the entire service lifecycle: from design, through the development process, and then into production support Specifically DevOps is characterized by: Operations staff making use many of the same techniques as developers for their systems work Developers incorporating more operational concerns in their planning and coding (designing for operationalization) DevOps can be a team, a department, an effort, a movement, but most importantly it is a culture! KEY TENETS OF DEVOPS Operate Plan FEEDBACK Theory of Constraints (“The Goal”) Release Architect The 3 Ways (“The Phoenix Project”): Systems thinking Amplified feedback loops Test Dev Continuous experimentation and learning Lean principles of manufacturing (“TPS: Toyota Production System”) People, then process, then tools (Men, Methods, Machines, Measurements) Collaborative development (Scrum, Agile, Kanban) Environment provisioning, quicker access to new code for testing Continuous deployment, integration, testing, delivery, feedback, and improvement THE NEW DEVOPS CULTURE Automation Specialty Testing Testing Quality Assurance Release Release Management Automation Data Services Team Team DevOps Environment Application Management Reliability Planning (Enterprise) Agile Project Small Increased Speed to Defect Minimal Viable Build Quality Into Tracking Batches Frequency Resolution Product the Process Mapping Architecture, Design & Development Value Stream Stream Value Developer Application Design for Operations Loosely Coupled Micro Branching by Commit Instrumentation Incl. Graceful Degradation Architectures services Abstraction /Toggling The Automated Delivery Pipeline (DevOps) Continuous Environment Deployment Automated Testing Production Feedback Standardization Integration Management Orchestration Deployment Mechanisms Layered Testing (GUI, API, Version Baselined / Automated Backend) Deployment Dashboards/ Build Automation Modelled Deployment Full System Testing Phased Rollout Scorecards Basic Test (incl. Environments Development (Functional, UX Rollback Plans Trends static code quality, Test Data Integration Verification, Regression, APM & Analytics Reports Lean security, Propagation Acceptance Security, Performance, Communication Notifications Principles performance, and Automatic Performance Compatibility, ALDM code coverage) Configuration Production Acceptance) Monitor/Measure Package Infrastructure-as- On-Demand Environment Sanity Tests Everything Deploy to Software Code Environment Application Resiliency Continuous Process Asset Repository Automation Provisioning Testing (Chaos Monkey, etc.) Improvement Governance Checks On-Premises/ Non-Functional Sanity Common Metrics Feedback Cloud/Hybrid Tests Measuring Quality Infrastructure Resiliency Removal of of Removal Mechanism Virtualization Measuring Process Debt Process Infrastructure Loosely Coupled Testing Efficiency Integrations Architectures Visible Status Continuous Continuous Trust Dev in Ops Ops in Dev Common Goals based on Experimentation Improvement Customer Satisfaction Removal of of Removal Collaboration / Break Collective Blameless Openness & Technical Debt Technical Never Done Down Silos Ownership Culture (Foundational) Postmortem Visibility Planning (Enterprise) Agile Project Small Increased Speed to Defect Minimal Viable Build Quality Into Tracking Batches Frequency Resolution Product the Process Mapping Architecture, Design & Development Value Stream Stream Value Developer Application Design for Operations Loosely Coupled Micro Branching by Commit Instrumentation Incl. Graceful Degradation Architectures services Abstraction /Toggling The Automated Delivery Pipeline (DevOps) Continuous Environment Deployment Automated Testing Production Feedback Standardization Integration Management Orchestration Deployment Mechanisms Layered Testing (GUI, API, Version Baselined / Automated Backend) Deployment Dashboards/ Build Automation Modelled Deployment Full System Testing Phased Rollout Scorecards Basic Test (incl. Environments Development (Functional, UX Rollback Plans Trends static code quality, Test Data Integration Verification, Regression, APM & Analytics Reports Lean security, Propagation Acceptance Security, Performance, Communication Notifications Principles performance, and Automatic Performance Compatibility, ALDM code coverage) Configuration Production Acceptance) Monitor/Measure Package Infrastructure-as- On-Demand Environment Sanity Tests Everything Deploy to Software Code Environment Application Resiliency Continuous Process Asset Repository Automation Provisioning Testing (Chaos Monkey, etc.) Improvement Governance Checks On-Premises/ Non-Functional Sanity Common Metrics Feedback Cloud/Hybrid Tests Measuring Quality Infrastructure Resiliency Removal of of Removal Mechanism Virtualization Measuring Process Debt Process Infrastructure Loosely Coupled Testing Efficiency Integrations Architectures Visible Status Continuous Continuous Trust Dev in Ops Ops in Dev Common Goals based on Experimentation Improvement Customer Satisfaction Removal of of Removal Collaboration / Break Collective Blameless Openness & Technical Debt Technical Never Done Down Silos Ownership Culture (Foundational) Postmortem Visibility NORMING: WHO DOES WHAT? The Automated Delivery Pipeline (DevOps) Continuous Environment Deployment Automated Testing Production Feedback Integration Management Orchestration Deployment Mechanisms Layered Testing (GUI, API, Version Baselined / Automated Backend) Deployment Dashboards/ Build Automation Modelled Deployment Full System Testing Phased Rollout Scorecards Basic Test (incl. Environments Development (Functional, UX Rollback Plans Trends static code quality, Test Data Integration Verification, Regression, APM & Analytics Reports security, Propagation Acceptance Security, Performance, Communication Notifications performance, and Automatic Performance Compatibility, ALDM code coverage) Configuration Production Acceptance) Monitor/Measure Package Infrastructure-as- On-Demand Environment Sanity Tests Everything Deploy to Software Code Environment Application Resiliency Continuous Process Asset Repository Automation Provisioning Testing (Chaos Monkey, etc.) Improvement
Recommended publications
  • Core Elements of Continuous Testing

    Core Elements of Continuous Testing

    WHITE PAPER CORE ELEMENTS OF CONTINUOUS TESTING Today’s modern development disciplines -- whether Agile, Continuous Integration (CI) or Continuous Delivery (CD) -- have completely transformed how teams develop and deliver applications. Companies that need to compete in today’s fast-paced digital economy must also transform how they test. Successful teams know the secret sauce to delivering high quality digital experiences fast is continuous testing. This paper will define continuous testing, explain what it is, why it’s important, and the core elements and tactical changes development and QA teams need to make in order to succeed at this emerging practice. TABLE OF CONTENTS 3 What is Continuous Testing? 6 Tactical Engineering Considerations 3 Why Continuous Testing? 7 Benefits of Continuous Testing 4 Core Elements of Continuous Testing WHAT IS CONTINUOUS TESTING? Continuous testing is the practice of executing automated tests throughout the software development cycle. It’s more than just automated testing; it’s applying the right level of automation at each stage in the development process. Unlike legacy testing methods that occur at the end of the development cycle, continuous testing occurs at multiple stages, including development, integration, pre-release, and in production. Continuous testing ensures that bugs are caught and fixed far earlier in the development process, improving overall quality while saving significant time and money. WHY CONTINUOUS TESTING? Continuous testing is a critical requirement for organizations that are shifting left towards CI or CD, both modern development practices that ensure faster time to market. When automated testing is coupled with a CI server, tests can instantly be kicked off with every build, and alerts with passing or failing test results can be delivered directly to the development team in real time.
  • Studying the Feasibility and Importance of Software Testing: an Analysis

    Studying the Feasibility and Importance of Software Testing: an Analysis

    Dr. S.S.Riaz Ahamed / Internatinal Journal of Engineering Science and Technology Vol.1(3), 2009, 119-128 STUDYING THE FEASIBILITY AND IMPORTANCE OF SOFTWARE TESTING: AN ANALYSIS Dr.S.S.Riaz Ahamed Principal, Sathak Institute of Technology, Ramanathapuram,India. Email:[email protected], [email protected] ABSTRACT Software testing is a critical element of software quality assurance and represents the ultimate review of specification, design and coding. Software testing is the process of testing the functionality and correctness of software by running it. Software testing is usually performed for one of two reasons: defect detection, and reliability estimation. The problem of applying software testing to defect detection is that software can only suggest the presence of flaws, not their absence (unless the testing is exhaustive). The problem of applying software testing to reliability estimation is that the input distribution used for selecting test cases may be flawed. The key to software testing is trying to find the modes of failure - something that requires exhaustively testing the code on all possible inputs. Software Testing, depending on the testing method employed, can be implemented at any time in the development process. Keywords: verification and validation (V & V) 1 INTRODUCTION Testing is a set of activities that could be planned ahead and conducted systematically. The main objective of testing is to find an error by executing a program. The objective of testing is to check whether the designed software meets the customer specification. The Testing should fulfill the following criteria: ¾ Test should begin at the module level and work “outward” toward the integration of the entire computer based system.
  • Customer Success Story

    Customer Success Story

    Customer Success Story Interesting Dilemma, Critical Solution Lufthansa Cargo AG The purpose of Lufthansa Cargo AG’s SDB Lufthansa Cargo AG ordered the serves more than 500 destinations world- project was to provide consistent shipment development of SDB from Lufthansa data as an infrastructure for each phase of its Systems. However, functional and load wide with passenger and cargo aircraft shipping process. Consistent shipment data testing is performed at Lufthansa Cargo as well as trucking services. Lufthansa is is a prerequisite for Lufthansa Cargo AG to AG with a core team of six business one of the leaders in the international air efficiently and effectively plan and fulfill the analysts and technical architects, headed cargo industry, and prides itself on high transport of shipments. Without it, much is at by Project Manager, Michael Herrmann. stake. quality service. Herrmann determined that he had an In instances of irregularities caused by interesting dilemma: a need to develop inconsistent shipment data, they would central, stable, and optimal-performance experience additional costs due to extra services for different applications without handling efforts, additional work to correct affecting the various front ends that THE CHALLENGE accounting information, revenue loss, and were already in place or currently under poor feedback from customers. construction. Lufthansa owns and operates a fleet of 19 MD-11F aircrafts, and charters other freight- With such critical factors in mind, Lufthansa Functional testing needed to be performed Cargo AG determined that a well-tested API on services that were independent of any carrying planes. To continue its leadership was the best solution for its central shipment front ends, along with their related test in high quality air cargo services, Lufthansa database.
  • View Corporate and Individual Cultures in a Setting Or of Brands and Trademarks, There Is Also a Sharp Irrevocable Openness [26]

    View Corporate and Individual Cultures in a Setting Or of Brands and Trademarks, There Is Also a Sharp Irrevocable Openness [26]

    Teixeira and Karsten Journal of Internet Services and Applications (2019) 10:7 Journal of Internet Services https://doi.org/10.1186/s13174-019-0105-z and Applications RESEARCH Open Access Managing to release early, often and on time in the OpenStack software ecosystem José Apolinário Teixeira* and Helena Karsten Abstract The dictum of “Release early, release often.” by Eric Raymond as the Linux modus operandi highlights the importance of release management in open source software development. However, there are very few empirical studies addressing release management in this context. It is already known that most open source software communities adopt either a feature-based or time-based release strategy. Both have their own advantages and disadvantages that are also context-specific. Recent research reports that many prominent open source software projects have overcome a number of recurrent problems by moving from feature-based to time-based release strategies. In this longitudinal case study, we address the release management practices of OpenStack, a large scale open source project developing cloud computing technologies. We discuss how the release management practices of OpenStack have evolved in terms of chosen strategy and timeframes with close attention to processes and tools. We discuss the number of practical and managerial issues related to release management within the context of large and complex software ecosystems. Our findings also reveal that multiple release management cycles can co-exist in large and complex software
  • Magic Quadrant for Application Release Automation

    Magic Quadrant for Application Release Automation

    Magic Quadrant for Application Release Automation Published: 27 September 2017 ID: G00315074 Analyst(s): Colin Fletcher, Laurie F. Wurster The ARA market is rapidly evolving in response to growing enterprise requirements to both scale DevOps initiatives and improve release management agility across multiple cultures, processes and generations of technology. This research helps I&O leaders make better-informed decisions. Strategic Planning Assumption By 2020, 50% of global enterprises will have implemented at least one application release automation solution, up from less than 15% today. Market Definition/Description Application release automation (ARA) tools provide a combination of automation, environment modeling and release coordination capabilities to simultaneously improve the quality and the velocity of application releases. ARA tools enable best practices in moving application-related artifacts, applications, configurations and even data together across the application life cycle process. These tools are a key part of enabling the DevOps goal of achieving continuous delivery with large numbers of rapid, small releases. Approximately seven years old, the ARA solution market reached an estimated $228.2 million in 2016, up 31.4% from $173.6 million in 2015. The market is continues to be expected to grow at an estimated 20% compound annual growth rate (CAGR) through 2020. Magic Quadrant Figure 1. Magic Quadrant for Application Release Automation Source: Gartner (September 2017) Vendor Strengths and Cautions Arcad Software Founded in 1992, Arcad Software is a privately held company headquartered in Chavanod, France. The company was started by its founder to deliver automation-oriented solutions supporting the Page 2 of 28 Gartner, Inc. | G00315074 IBM i (introduced as AS/400, then later renamed eServer iSeries) platform.
  • Leading Practice: Test Strategy and Approach in Agile Projects

    Leading Practice: Test Strategy and Approach in Agile Projects

    CA SERVICES | LEADING PRACTICE Leading Practice: Test Strategy and Approach in Agile Projects Abstract This document provides best practices on how to strategize testing CA Project and Portfolio Management (CA PPM) in an agile project. The document does not include specific test cases; the list of test cases and steps for each test case are provided in a separate document. This document should be used by the agile project team that is planning the testing activities, and by end users who perform user acceptance testing (UAT). Concepts Concept Description Test Approach Defines testing strategy, roles and responsibilities of various team members, and test types. Testing Environments Outlines which testing is carried out in which environment. Testing Automation and Tools Addresses test management and automation tools required for test execution. Risk Analysis Defines the approach for risk identification and plans to mitigate risks as well as a contingency plan. Test Planning and Execution Defines the approach to plan the test cases, test scripts, and execution. Review and Approval Lists individuals who should review, approve and sign off on test results. Test Approach The test approach defines testing strategy, roles and responsibilities of various team members, and the test types. The first step is to define the testing strategy. It should describe how and when the testing will be conducted, who will do the testing, the type of testing being conducted, features being tested, environment(s) where the testing takes place, what testing tools are used, and how are defects tracked and managed. The testing strategy should be prepared by the agile core team.
  • Functional Testing Functional Testing

    Functional Testing Functional Testing

    From Pressman, “Software Engineering – a practitionerʼs approach”, Chapter 14 and Pezze + Young, “Software Testing and Analysis”, Chapters 10-11 Today, weʼll talk about testing – how to test software. The question is: How do we design tests? And weʼll start with Functional Testing functional testing. Software Engineering Andreas Zeller • Saarland University 1 Functional testing is also called “black- box” testing, because we see the program as a black box – that is, we ignore how it is being written 2 in contrast to structural or “white-box” testing, where the program is the base. 3 If the program is not the base, then what is? Simple: itʼs the specification. 4 If the program is not the base, then what is? Simple: itʼs the specification. Testing Tactics Functional Structural “black box” “white box” • Tests based on spec • Tests based on code • Test covers as much • Test covers as much specified behavior implemented behavior as possible as possible 5 Why Functional? Functional Structural “black box” “white box” • Program code not necessary • Early functional test design has benefits reveals spec problems • assesses testability • gives additional explanation of spec • may even serve as spec, as in XP 6 Structural testing can not detect that some required feature is missing in the code Why Functional? Functional testing applies at all granularity levels (in contrast to structural testing, which only applies to Functional Structural unit and integration testing) “black box” “white box” • Best for missing logic defects Common problem: Some program logic was simply forgotten Structural testing would not focus on code that is not there • Applies at all granularity levels unit tests • integration tests • system tests • regression tests 7 2,510,588,971 years, 32 days, and 20 hours to be precise.
  • HP Functional Testing Software Data Sheet

    HP Functional Testing Software Data Sheet

    HP Functional Testing software Data sheet With HP Functional Testing you can automate functional and regression testing for every modern software application and environment, extend testing to a wider range of teams, and accelerate the testing process—so you can improve application quality and still make your market window. Simplifies test creation and HP Functional Testing makes it easy to insert, modify, data-drive, and remove test steps. It features: maintenance • Keyword capabilities: Using keywords, testers HP Functional Testing is advanced, automated can build test cases by capturing flows directly testing software for building functional and regression from the application screens and applying robust test suites. It captures, verifies, and replays user record/replay capturing technology. interactions automatically and helps testers quickly • Automatic updating: With new application builds, identify and report on application effects, while you only need to update one reference in the shared providing sophisticated functionality for tester repository and the update is propagated to all collaboration. The product includes HP QuickTest referencing tests. Professional and all of its add-ins. It is sold stand-alone or as part of the broader HP Unified • Easy data-driving: You can quickly data-drive any Functional Testing solution, which couples object definition, method, checkpoint, and output HP Functional Testing with HP Service Test to value through the integrated data table. address both GUI and non-GUI testing. • Timely advice: In
  • Devsecops DEVELOPMENT & DEVOPS INFRASTRUCTURE

    Devsecops DEVELOPMENT & DEVOPS INFRASTRUCTURE

    DevSecOps DEVELOPMENT & DEVOPS INFRASTRUCTURE CREATE SECURE APPLICATIONS PARASOFT’S APPROACH - BUILD SECURITY IN WITHOUT DISRUPTING THE Parasoft provides tools that help teams begin their security efforts as DEVELOPMENT PROCESS soon as the code is written, starting with static application security test- ing (SAST) via static code analysis, continuing through testing as part of Parasoft makes DevSecOps possible with API and the CI/CD system via dynamic application security testing (DAST) such functional testing, service virtualization, and the as functional testing, penetration testing, API testing, and supporting in- most complete support for important security stan- frastructure like service virtualization that enables security testing be- dards like CWE, OWASP, and CERT in the industry. fore the complete application is fully available. IMPLEMENT A SECURE CODING LIFECYCLE Relying on security specialists alone prevents the entire DevSecOps team from securing software and systems. Parasoft tooling enables the BENEFIT FROM THE team with security knowledge and training to reduce dependence on PARASOFT APPROACH security specialists alone. With a centralized SAST policy based on in- dustry standards, teams can leverage Parasoft’s comprehensive docs, examples, and embedded training while the code is being developed. ✓ Leverage your existing test efforts for Then, leverage existing functional/API tests to enhance the creation of security security tests – meaning less upfront cost, as well as less maintenance along the way. ✓ Combine quality and security to fully understand your software HARDEN THE CODE (“BUILD SECURITY IN”) Getting ahead of application security means moving beyond just test- ✓ Harden the code – don’t just look for ing into building secure software in the first place.
  • Continuous Quality and Testing to Accelerate Application Development

    Continuous Quality and Testing to Accelerate Application Development

    Continuous Quality and Testing to Accelerate Application Development How to assess your current testing maturity level and practice continuous testing for DevOps Continuous Quality and Testing to Accelerate Application Development // 1 Table of Contents 03 Introduction 04 Why Is Continuous Quality and Testing Maturity Important to DevOps? 05 Continuous Testing Engineers Quality into DevOps 07 Best Practices for Well- Engineered Continuous Testing 08 Continuous Testing Maturity Levels Level 1: Chaos Level 2: Continuous Integration Level 3: Continuous Flow Level 4: Continuous Feedback Level 5: Continuous Improvement 12 Continuous Testing Maturity Assessment 13 How to Get Started with DevOps Testing? 14 Continuous Testing in the Cloud Choosing the right tools for Continuous Testing On-demand Development and Testing Environments with Infrastructure as Code The Right Tests at the Right Time 20 Get Started 20 Conclusion 21 About AWS Marketplace and DevOps Institute 21 Contributors Introduction A successful DevOps implementation reduces the bottlenecks related to testing. These bottlenecks include finding and setting up test environments, test configurations, and test results implementation. These issues are not industry specific. They can be experienced in manufacturing, service businesses, and governments alike. They can be reduced by having a thorough understanding and a disciplined, mature implementation of Continuous Testing and related recommended engineering practices. The best place to start addressing these challenges is having a good understanding of what Continuous Testing is. Marc Hornbeek, the author of Engineering DevOps, describes it as: “A quality assessment strategy in which most tests are automated and integrated as a core and essential part of DevOps. Continuous Testing is much more than simply ‘automating tests.’” In this whitepaper, we’ll address the best practices you can adopt for implementing Continuous Quality and Testing on the AWS Cloud environment in the context of the DevOps model.
  • Chap 3. Test Models and Strategies 3.5 Test Integration and Automation 1

    Chap 3. Test Models and Strategies 3.5 Test Integration and Automation 1

    Chap 3. Test Models and Strategies 3.5 Test Integration and Automation 1. Introduction 2. Integration Testing 3. System Testing 4. Test Automation Appendix: JUnit Overview 1 1. Introduction -A system is composed of components. System of software components can be defined at any physical scope. Component System Typical intercomponent interfaces (locus of (Focus of Integration) (Scope of integration faults) Integration) Method Class Instance variables Intraclass messages Class Cluster Intraclass messages Cluster Subsystem Interclass messages Interpackage messages Subsystem System Inteprocess communication Remote procedure call ORB services, OS services -Integration test design is concerned with several primary questions: 1. Which components and interfaces should be exercised? 2. In what sequence will component interfaces be exercised? 2 3. Which test design technique should be used to exercise each interface? -Integration testing is a search for component faults that cause intercomponent failures. -System scope testing is a search for faults that lead to a failure to meet a system scope responsibility. ÷System scope testing cannot be done unless components interoperate sufficiently well to exercise system scope responsibilities. -Effective testing at system scope requires a concrete and testable system-level specification. ÷System test cases must be derived from some kind of functional specification. Traditionally, user documentation, product literature, line-item narrative requirements, and system scope models have been used. 3 2. Integration Testing -Unit testing focuses on individual components. Once faults in each component have been removed and the test cases do not reveal any new fault, components are ready to be integrated into larger subsystems. -Integration testing detects faults that have not been detected during unit testing, by focusing on small groups of components.
  • Integration Testing of Object-Oriented Software

    Integration Testing of Object-Oriented Software

    POLITECNICO DI MILANO DOTTORATO DI RICERCA IN INGEGNERIA INFORMATICA E AUTOMATICA Integration Testing of Object-Oriented Software Ph.D. Thesis of: Alessandro Orso Advisor: Prof. Mauro Pezze` Tutor: Prof. Carlo Ghezzi Supervisor of the Ph.D. Program: Prof. Carlo Ghezzi XI ciclo To my family Acknowledgments Finding the right words and the right way for expressing acknowledgments is a diffi- cult task. I hope the following will not sound as a set of ritual formulas, since I mean every single word. First of all I wish to thank professor Mauro Pezze` , for his guidance, his support, and his patience during my work. I know that “taking care” of me has been a hard work, but he only has himself to blame for my starting a Ph.D. program. A very special thank to Professor Carlo Ghezzi for his teachings, for his willingness to help me, and for allowing me to restlessly “steal” books and journals from his office. Now, I can bring them back (at least the one I remember...) Then, I wish to thank my family. I owe them a lot (and even if I don't show this very often; I know this very well). All my love goes to them. Special thanks are due to all my long time and not-so-long time friends. They are (stricty in alphabetical order): Alessandro “Pari” Parimbelli, Ambrogio “Bobo” Usuelli, Andrea “Maken” Machini, Antonio “the Awesome” Carzaniga, Dario “Pitone” Galbiati, Federico “Fede” Clonfero, Flavio “Spadone” Spada, Gianpaolo “the Red One” Cugola, Giovanni “Negroni” Denaro, Giovanni “Muscle Man” Vigna, Lorenzo “the Diver” Riva, Matteo “Prada” Pradella, Mattia “il Monga” Monga, Niels “l’e´ semper chi” Kierkegaard, Pierluigi “San Peter” Sanpietro, Sergio “Que viva Mex- ico” Silva.