DOCSLIB.ORG

Continuous API Testing and Monitoring: Best Practices & Buy-In

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Continuous API Testing and Monitoring: Best Practices & Buy-in Guide Evaluate continuous API testing and monitoring tools to significantly improve API quality, security, and reliability v2.1 Contents Four Key API Testing Strategies 4 1. Use Dynamic DDT to Check Entire User Flows 4 2. Ensure API Tests Are Created with Domain Expertise 5 3. Invest in an API Testing Tool with a Modern Architecture 5 4. Adopt a New API Metric: E2E Functional Uptime 5 Four Important API Testing Solutions 6 Solution 1: Solve QA/Testing Bottlenecks with Transparent API Testing 6 Solution 2: Monitor APIs With or Without a CI/CD Pipeline 7 Solution 3: Reuse API Tests as Functional Uptime Monitors in Production 8 Solution 4: Create a Single Pane of Performance Analytics 9 API Testing Automation: Evaluation Criteria 11 Features Evaluation Checklist 11 Conclusion 12 About API Fortress 13 Continuous API Testing and Monitoring: Best Practices & Buy-in Guide 2 Shift-left API testing automation has been increasingly embraced by developers and QA teams as essential to successful agile software development or a CI/CD pipeline. In theory, automated testing early in the lifecycle should reduce costly late-lifecycle QA/testing bottlenecks. Yet bottlenecks continue to plague developers and testers, and the number of undetected API bugs that go live and reach end-users or hackers remains alarmingly high. $2.8 trillion was spent in the U.S. due to poor quality software (CISQ) One of the critical issues behind a failure of testing effectiveness at agile organizations involves the complexity of managing the scope and scale of API testing, that is, in striking the right balance between time-to-market and quality-to-market. Companies need to do more testing in less time, but a major roadblock stands in the way - the diverse teams that share in API success tend to work in silos. Development, QA/Testing, and DevOps teams may struggle to find consensus on which API metrics should be tracked to set KPIs A core reason for silos in an agile organization stems from the wide gap in priorities and accountability among the teams that own APIs. While developers may be most concerned about delivering a shippable product on time, QA/testing teams may be more concerned about validating whether the product satisfies the user story. Then DevOps or DevSecOps leaders have uptime and reliability concerns that extend beyond the delivery sprint. Banking, financial services, healthcare, and other types of enterprises that deal with advanced API security and compliance issues run into more complications that divide siloed teams even further. With the right API testing and monitoring tools, agile organizations can bring insight-driven visibility and collaboration to distributed teams. This is needed to set companies on the right path to transform API testing reliability and performance. This white paper offers a breakdown of recommended API testing strategies along with insights about using the right API metrics to set up more viable KPIs based on our years of experience at API Fortress in working with several of the world’s most innovative retail, banking, financial services, insurance, healthcare, telecom, education, entertainment, and aeronautics/aviation companies. Additionally, this white paper includes a breakdown of best practices for implementing good API testing and monitoring tools. Continuous API Testing and Monitoring: Best Practices & Buy-in Guide 3 Four Key API Testing Strategies Prior to agile development, CI/CD pipelines and microservices, UI and Unit testing dominated the testing pyramid. And then, the only essential role of API testing was to verify Test Automation the contract and check for a ping round trip. Now, API tests must effectively act as end- to-end tests that validate entire user stories. UI Tests This new set of strategies (along with new metrics) are needed to determine API testing E2E Testing success. The following “Big Four” strategies have been derived from what we have seen in next-generation approaches to API testing for modern systems with contemporary toolchains and CICD flows. We hope these strategies help leaders of development, QA and testing, enterprise architecture, DevOps, and product teams to better quantify the efficiency and efficacy of their current or proof-of-concept API testing tools. Strategy 1: Use Dynamic DDT to Check Entire User Flows Modern API testing tools must be able to conduct multi-step integration tests across APIs to check on how well API endpoints work together (the “API flow”). In the real world, APIs use data that is not fixed or prebuilt. It follows that API testing tools should create API testing paths that are unpredictable. The best way to do this is to leverage Data-driven Testing (DDT) from active databases or highly variable (fake) testing data. The goal is to create tests that involve multiple steps, including calling an API that can trigger a sequence of unique API calls in an array. In some cases, testing systems include components that convert databases into APIs to be used as these dynamic APIs. Additionally, modern API testing tools should automate OAuth 2.0 flows to avoid disruption to DDT while also validating SSO and multi-factor authorizations. Continuous API Testing and Monitoring: Best Practices & Buy-in Guide 4 Strategy 2: Ensure API Tests Are Created with Domain Expertise While developers can certainly write tests that verify the technical capabilities of the APIs they build, it can be ineffective and/or inefficient for developers to create API tests that also validate the user story. Instead, API tests from developers that simply prove whether an API is working when used correctly can form the basis for more stringent and comprehensive tests written by professional QAs with high knowledge of the problem and solution domains. These holistic tests may include end-to-end integration tests that can check entire flows involving arrays of APIs that may have been built or changed by many different teams. Ultimately, the goal is to leverage one unified test with consistent domain knowledge for proactive and real- time insight about API health throughout constant changes to code and databases. Strategy 3: Invest in an API Testing Tool with a Modern Architecture Modern API testing tools, in particular, tools that excel at testing APIs in agile development and CI/CD pipelines, must offer a robust set of APIs for plug-and-play integrations with existing and modern DevOps tools. Today, many companies benefit from sending all test results to a best-in-breed analytics platform such as Elastic or Splunk, while delegating notifications to best-in-breed solutions such as PagerDuty or Slack. Additionally, with modern architectures, API testing tools can evolve along a far more innovative roadmap. If a company is evaluating a number of API testing tools that are divided between unified testing suites that piecemeal together multiple apps/services versus a best- in-breed unified testing platform, the best-in-breed platform is almost always the better choice. Most API owners should avoid vendor lock-in, and future-proof their increasingly agile toolchains for cloud maturity and distributed services. Strategy 4: Adopt a New API Metric: E2E Functional Uptime On its own, API uptime (an HTTP 200 OK) is an outmoded metric for continuous API quality. Modern API testing tools must go beyond uptime, and simplify validation of the user story by checking the business logic and service layers for problems across functionality, reliability, performance, and compliance as well as potential security vulnerabilities. A holistic verification of these layers allows a good API testing tool to significantly improve on the accuracy and consistency of uptime reporting, particularly, in reducing false-positives and ensuring adequate testing coverage. Testing Centers of Excellence or smaller QA/Testing teams sometimes refer to this new API metric as “Functional Uptime” or “End-to-End (E2E) Uptime.” Continuous API Testing and Monitoring: Best Practices & Buy-in Guide 5 Four Important API Testing Solutions Before breaking down the best practices of a good API testing tool, this section provides a framework for how the best practices were selected. The focus of the assessment is geared toward solving four key API testing solutions that are critical for any organization moving from a monolithic stack to distributed services and cloud maturity. Solution 1: Solve QA/Testing Bottlenecks with Transparent API Testing 63 32 22 23 Plan Build Test/QA Release / Deploy Most of the costly and time-consuming bottlenecks that hold up software development happen in the QA/testing stage. Insufficient or poor collaboration between technical and product teams (or line of business owners) with poor or no visibility into end-to-end API testing is often at fault. True end-to-end API testing tools tackle this problem by making it easy for stakeholders of all technical and coding backgrounds to work in parallel. Ultimately, the goal is to minimize the risk of falling short of validating the user story without delaying go-to-market. Another vital aspect of API testing transparency is to ensure that all teams are clear on the scope of testing coverage. Insufficient attention to this detail may result in high numbers of false-positives, which allow malfunctioning APIs to exist in production environments for prolonged periods of time. Continuous API Testing and Monitoring: Best Practices & Buy-in Guide 6 Solution 2: Monitor APIs With or Without a CI/CD Pipeline Continuous API Testing and Monitoring in a CI Flow Developers & QA CI/CD Pipeline Centralized API Testing and Debugging Repo Command Line Interface Test Eecution esults IE Iocal Successful shift-left API testing automation frees development teams to confidently run a Continuous Integration (CI) flow. Most API testing tools simplify the CI flow by seamlessly integrating with popular CI/CD platforms such as Jenkins, BitBucket, Azure DevOps, Bamboo, TravisCI, GitLab CI, CircleCI and more.
Recommended publications
  • Core Elements of Continuous Testing

    Core Elements of Continuous Testing

    WHITE PAPER CORE ELEMENTS OF CONTINUOUS TESTING Today’s modern development disciplines -- whether Agile, Continuous Integration (CI) or Continuous Delivery (CD) -- have completely transformed how teams develop and deliver applications. Companies that need to compete in today’s fast-paced digital economy must also transform how they test. Successful teams know the secret sauce to delivering high quality digital experiences fast is continuous testing. This paper will define continuous testing, explain what it is, why it’s important, and the core elements and tactical changes development and QA teams need to make in order to succeed at this emerging practice. TABLE OF CONTENTS 3 What is Continuous Testing? 6 Tactical Engineering Considerations 3 Why Continuous Testing? 7 Benefits of Continuous Testing 4 Core Elements of Continuous Testing WHAT IS CONTINUOUS TESTING? Continuous testing is the practice of executing automated tests throughout the software development cycle. It’s more than just automated testing; it’s applying the right level of automation at each stage in the development process. Unlike legacy testing methods that occur at the end of the development cycle, continuous testing occurs at multiple stages, including development, integration, pre-release, and in production. Continuous testing ensures that bugs are caught and fixed far earlier in the development process, improving overall quality while saving significant time and money. WHY CONTINUOUS TESTING? Continuous testing is a critical requirement for organizations that are shifting left towards CI or CD, both modern development practices that ensure faster time to market. When automated testing is coupled with a CI server, tests can instantly be kicked off with every build, and alerts with passing or failing test results can be delivered directly to the development team in real time.
  • Drive Continuous Delivery with Continuous Testing

    Drive Continuous Delivery with Continuous Testing

    I Don’t Believe Your Company Is Agile! Alex Martins CTO Continuous Testing CA Technologies October 2017 1 Why Many Companies Think They’re Agile… They moved some Dev projects from waterfall to agile They’re having daily standups They have a scrum master Product owner is part of the team They are all talking and walking agile… And are talking about Continuous Delivery BUT… QA is STILL a Bottleneck… Even in DevOps Shops A 2017 survey of self- …of delays were occurring at proclaimed DevOps 63% the Test/QA stage of the practitioners found that … cycle. “Where are the main hold-ups in the software production process?” 63% 32% 30% 16% 22% 21% 23% http://www.computing.co.uk/digital_assets/634fe325-aa28-41d5-8676-855b06567fe2/CTG-DevOps-Review-2017.pdf 3 Challenges to Achieving Continuous Delivery & Testing IDEA Requirements 64% of total defect cost originate in the requirements analysis and design phase1. ? ? of developers time is spent 80% of teams experience delays in development and QA Development 50% 3 finding and fixing defects2 due to unavailable dependencies X X Security 70x required manual pen 30% of teams only security scan 50% more time spent on security test scan cost vs. automation10 once per year9 defects in lower-performing teams8 X X X 70% of all 63% of testers admit they 50% of time 79% of teams face prohibitive QA / Testing testing is still can’t test across all the different spent looking for restrictions, time limits or access manual4 devices and OS versions5 test data6 fees on needed 3rd party services3 ! Release 57% are dissatisfied with the time it takes to deploy new features7 ! ! Ave.
  • Your Continuous Testing with Spirent's Automation Platforms

    Your Continuous Testing with Spirent's Automation Platforms

    STREAMLINE Your Continuous Testing With Spirent’s Automation Platforms DE RELE VE AS LO E P E T A ING T R G ES E T T S N U I O U IN T N E CO STAG Table of Contents Agile 3 What is Continuous Test (CT)? 4 What Hinders CT Implementation? 5 The Chasm 6 The Bridge 7 Spirent Velocity Framework 8 Lab as a Service (LaaS) 9 Test as a Service (TaaS) 10 CT Implementation Best Practices 11 Summary 12 2 of 12 Agile software development practices gained momentum in the late 90s. Agile emphasizes close collaboration between business stakeholders, the development team, and QA. This enabled faster software delivery, better quality and improved customer satisfaction. By employing DevOps practices, the pace and benefits are amplified. 3 of 12 What is Continuous Test (CT)? Continuous Test (CT) enables CT haed Cotuous terato ad Deery ee network testing to be more effectively performed by DEVELOP ITEATE STAGE ELEASE development teams by enabling them to take advantage of the QA team’s knowledge of real world customer use cases and environments. This is known as “shift left” because testing is moved earlier in the development cycle. With “shift left” tests are run as early as possible to accelerate understanding of AUTOMATE TESTS ORCHESTRATE TEST ENVIRONMENTSEXECUTE TESTS EARLIER problem areas in the code and where development attention is required. Why Do You Need CT? The combination of earlier and faster testing shortens time to release while improving quality and customer satisfaction. 4 of 12 What Hinders CT Implementation? Deficient or non existent Insufficient test Lack of test results Inadequate understanding of tools for creating resources for timely analysis tools hinder customer environments and automated tests test execution assessment of test results use cases by developers EW SOFTWAE DEVELOP ITEATE STAGE ELEASE SOFTWAE BILD ELEASE The promise of DevOps can’t be fully realized until Continuous Testing (CT) is factored in.
  • Model-Based Api Testing for Smt Solvers

    Model-Based Api Testing for Smt Solvers

    MODEL-BASED API TESTING FOR SMT SOLVERS Aina Niemetz ?y, Mathias Preiner ?y, Armin Biere ? ?Johannes Kepler University, Linz, Austria yStanford University, USA SMT Workshop 2017, July 22 – 23 Heidelberg, Germany SMT Solvers highly complex usually serve as back-end to some application key requirements: correctness robustness performance −! full verification difficult and still an open question −! solver development relies on traditional testing techniques 1/22 Testing of SMT Solvers State-of-the-art: unit tests regression test suite grammar-based black-box input fuzzing with FuzzSMT [SMT’09] generational input fuzzer for SMT-LIB v1 patched for SMT-LIB v2 compliance generates random but valid SMT-LIB input especially effective in combination with delta debugging not possible to test solver features not supported by the input language This work: model-based API fuzz testing −! generate random valid API call sequences 2/22 Model-Based API fuzz testing −! generate random valid API call sequences Previously: model-based API testing framework for SAT [TAP’13] implemented for the SAT solver Lingeling allows to test random solver configurations (option fuzzing) allows to replay erroneous solver behavior −! results promising for other solver back-ends Here: model-based API testing framework for SMT lifts SAT approach to SMT implemented for the SMT solver Boolector tailored to Boolector for QF_(AUF)BV with non-recursive first-order lambda terms −! effective and promising for other SMT solvers −! more general approach left to future
  • Continuous Testing Report 2019

    Continuous Testing Report 2019

    Continuous Testing Report In association with 2 Continuous Testing Report Contents Introduction 4 Executive summary 6 Current trends in continuous testing Test design 9 Functional and performance testing 13 Test data management 17 Test environment management 21 Test orchestration in the agile enterprise 24 Continuous testing: the road ahead 27 About the study 31 About the sponsors 37 3 Introduction Welcome dear readers. dependency on IT solutions today, with the integration of front-office and consumer-facing apps with back- Quality and testing approaches, methods, and office core systems, the leveraging of cloud and expertise have undergone radical changes over the microservices and the integration and use of IoT. And, last few years. Every organization today aspires on top of that, AI is emerging to make these solutions to deliver faster and more valuable IT solutions to autonomous and self learning. business and customers. To do this, they have been leveraging agile and DevOps methodologies and All this technology is delivered by different teams, using smarter automation technologies and as-a- many of which may not even be part of a single Service solutions to deliver IT faster and with greater company. flexibility. As we scramble to deliver innovative solutions for the At the same time, the IT landscape has also been newer, more complex IT landscape, there is, of course, growing in complexity. There is an increased a risk of failure. While some failures are inevitable and often provide a valuable learning opportunity (given a quick feedback loop), there are others that we must prevent from happening. Failures in core systems that seriously disrupt the business operations of an enterprise, failures that seriously impact a large number of clients and therefore jeopardize an organization’s reliability and brand perception, or failures in systems that cannot easily be rolled back all demand good testing of these systems before being deployed.
  • Devsecops DEVELOPMENT & DEVOPS INFRASTRUCTURE

    Devsecops DEVELOPMENT & DEVOPS INFRASTRUCTURE

    DevSecOps DEVELOPMENT & DEVOPS INFRASTRUCTURE CREATE SECURE APPLICATIONS PARASOFT’S APPROACH - BUILD SECURITY IN WITHOUT DISRUPTING THE Parasoft provides tools that help teams begin their security efforts as DEVELOPMENT PROCESS soon as the code is written, starting with static application security test- ing (SAST) via static code analysis, continuing through testing as part of Parasoft makes DevSecOps possible with API and the CI/CD system via dynamic application security testing (DAST) such functional testing, service virtualization, and the as functional testing, penetration testing, API testing, and supporting in- most complete support for important security stan- frastructure like service virtualization that enables security testing be- dards like CWE, OWASP, and CERT in the industry. fore the complete application is fully available. IMPLEMENT A SECURE CODING LIFECYCLE Relying on security specialists alone prevents the entire DevSecOps team from securing software and systems. Parasoft tooling enables the BENEFIT FROM THE team with security knowledge and training to reduce dependence on PARASOFT APPROACH security specialists alone. With a centralized SAST policy based on in- dustry standards, teams can leverage Parasoft’s comprehensive docs, examples, and embedded training while the code is being developed. ✓ Leverage your existing test efforts for Then, leverage existing functional/API tests to enhance the creation of security security tests – meaning less upfront cost, as well as less maintenance along the way. ✓ Combine quality and security to fully understand your software HARDEN THE CODE (“BUILD SECURITY IN”) Getting ahead of application security means moving beyond just test- ✓ Harden the code – don’t just look for ing into building secure software in the first place.
  • Continuous Quality and Testing to Accelerate Application Development

    Continuous Quality and Testing to Accelerate Application Development

    Continuous Quality and Testing to Accelerate Application Development How to assess your current testing maturity level and practice continuous testing for DevOps Continuous Quality and Testing to Accelerate Application Development // 1 Table of Contents 03 Introduction 04 Why Is Continuous Quality and Testing Maturity Important to DevOps? 05 Continuous Testing Engineers Quality into DevOps 07 Best Practices for Well- Engineered Continuous Testing 08 Continuous Testing Maturity Levels Level 1: Chaos Level 2: Continuous Integration Level 3: Continuous Flow Level 4: Continuous Feedback Level 5: Continuous Improvement 12 Continuous Testing Maturity Assessment 13 How to Get Started with DevOps Testing? 14 Continuous Testing in the Cloud Choosing the right tools for Continuous Testing On-demand Development and Testing Environments with Infrastructure as Code The Right Tests at the Right Time 20 Get Started 20 Conclusion 21 About AWS Marketplace and DevOps Institute 21 Contributors Introduction A successful DevOps implementation reduces the bottlenecks related to testing. These bottlenecks include finding and setting up test environments, test configurations, and test results implementation. These issues are not industry specific. They can be experienced in manufacturing, service businesses, and governments alike. They can be reduced by having a thorough understanding and a disciplined, mature implementation of Continuous Testing and related recommended engineering practices. The best place to start addressing these challenges is having a good understanding of what Continuous Testing is. Marc Hornbeek, the author of Engineering DevOps, describes it as: “A quality assessment strategy in which most tests are automated and integrated as a core and essential part of DevOps. Continuous Testing is much more than simply ‘automating tests.’” In this whitepaper, we’ll address the best practices you can adopt for implementing Continuous Quality and Testing on the AWS Cloud environment in the context of the DevOps model.
  • Model-Based API Testing for SMT Solvers∗

    Model-Based API Testing for SMT Solvers∗

    Model-Based API Testing for SMT Solvers∗ Aina Niemetz, Mathias Preiner, and Armin Biere Institute for Formal Models and Verification Johannes Kepler University, Linz, Austria Abstract Verification back ends such as SMT solvers are typically highly complex pieces of soft- ware with performance, correctness and robustness as key requirements. Full verification of SMT solvers, however, is difficult due to their complex nature and still an open question. Grammar-based black-box input fuzzing proved to be effective to uncover bugs in SMT solvers but is entirely input-based and restricted to a certain input language. State-of-the- art SMT solvers, however, usually provide a rich API, which often introduces additional functionality not supported by the input language. Previous work showed that applying model-based API fuzzing to SAT solvers is more effective than input fuzzing. In this pa- per, we introduce a model-based API testing framework for our SMT solver Boolector. Our experimental results show that model-based API fuzzing in combination with delta debugging techniques is effective for testing SMT solvers. 1 Introduction State-of-the-art Satisfiability Modulo Theories (SMT) solvers are typically highly complex pieces of software, and since they usually serve as back-end to some application, the level of trust in this application strongly depends on the level of trust in the underlying solver. Full verification of SMT solvers, however, is difficult due to their complex nature and still an open question. Hence, solver developers usually rely on traditional testing techniques such as unit and regression tests. At the SMT workshop in 2009, in [10] Brummayer et al.
  • Continuous Testing for Devops Evolving Beyond Simple Automation

    Continuous Testing for Devops Evolving Beyond Simple Automation

    Technical Whitepaper 1 Continuous Testing for DevOps Evolving Beyond Simple Automation INTRODUCTION DevOps represents a cultural shift that stresses collaboration be- on acceleration. Moreover, adopting a bona fide Continuous Testing tween the business, developers, and IT professionals. Software test process (more than just automated tests running regularly) helps automation can enhance these connections and help organizations promote all of the core pillars of DevOps: Culture, Automation, Lean, achieve desired SDLC acceleration, but it’s important to recognize Metrics, and Sharing. that automation is just one piece of the DevOps puzzle. In this paper, we’ll explore why and how Continuous Testing’s real- Since testing is often one of the greatest constraints in the SDLC, time objective assessment of an application’s business risks is a optimizing quality processes to allow testing to begin earlier, as well critical component of DevOps. as shrink the amount of testing required, can have a marked impact DEVOPS PRINCIPLES There are several key pieces to understanding DevOps revolutions and they are often brought about by a compelling event at an organization, such as a shift to agile. As organizations start to move into an agile development methodology, they start to uncover other processes that can be accelerated, such as delivery by DevOps and testing by Continuous Testing. The acceleration that is set in motion via agile makes it necessary to accelerate the release schedule. In order to ensure a successful release, an organization must adopt continuous testing to make sure the conveyer belt does not break down. The modernization maturity model has these three distinct phases: AGILE Agile software development is a different way of thinking about approaching the challenge of development time.
  • Fuzzing Radio Resource Control Messages in 5G and LTE Systems

    Fuzzing Radio Resource Control Messages in 5G and LTE Systems

    DEGREE PROJECT IN COMPUTER SCIENCE AND ENGINEERING, SECOND CYCLE, 30 CREDITS STOCKHOLM, SWEDEN 2021 Fuzzing Radio Resource Control messages in 5G and LTE systems To test telecommunication systems with ASN.1 grammar rules based adaptive fuzzer SRINATH POTNURU KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE Fuzzing Radio Resource Control messages in 5G and LTE systems To test telecommunication systems with ASN.1 grammar rules based adaptive fuzzer SRINATH POTNURU Master’s in Computer Science and Engineering with specialization in ICT Innovation, 120 credits Date: February 15, 2021 Host Supervisor: Prajwol Kumar Nakarmi KTH Supervisor: Ezzeldin Zaki Examiner: György Dán School of Electrical Engineering and Computer Science Host company: Ericsson AB Swedish title: Fuzzing Radio Resource Control-meddelanden i 5G- och LTE-system Fuzzing Radio Resource Control messages in 5G and LTE systems / Fuzzing Radio Resource Control-meddelanden i 5G- och LTE-system © 2021 Srinath Potnuru iii Abstract 5G telecommunication systems must be ultra-reliable to meet the needs of the next evolution in communication. The systems deployed must be thoroughly tested and must conform to their standards. Software and network protocols are commonly tested with techniques like fuzzing, penetration testing, code review, conformance testing. With fuzzing, testers can send crafted inputs to monitor the System Under Test (SUT) for a response. 3GPP, the standardiza- tion body for the telecom system, produces new versions of specifications as part of continuously evolving features and enhancements. This leads to many versions of specifications for a network protocol like Radio Resource Control (RRC), and testers need to constantly update the testing tools and the testing environment.
  • A Confused Tester in Agile World … Qa a Liability Or an Asset

    A Confused Tester in Agile World … Qa a Liability Or an Asset

    A CONFUSED TESTER IN AGILE WORLD … QA A LIABILITY OR AN ASSET THIS IS A WORK OF FACTS & FINDINGS BASED ON TRUE STORIES OF ONE & MANY TESTERS !! J Presented By Ashish Kumar, WHAT’S AHEAD • A STORY OF TESTING. • FROM THE MIND OF A CONFUSED TESTER. • FEW CASE STUDIES. • CHALLENGES IDENTIFIED. • SURVEY STUDIES. • GLOBAL RESPONSES. • SOLUTION APPROACH. • PRINCIPLES AND PRACTICES. • CONCLUSION & RECAP. • Q & A. A STORY OF TESTING IN AGILE… HAVE YOU HEARD ANY OF THESE ?? • YOU DON’T NEED A DEDICATED SOFTWARE TESTING TEAM ON YOUR AGILE TEAMS • IF WE HAVE BDD,ATDD,TDD,UI AUTOMATION , UNIT TEST >> WHAT IS THE NEED OF MANUAL TESTING ?? • WE WANT 100% AUTOMATION IN THIS PROJECT • TESTING IS BECOMING BOTTLENECK AND REASON OF SPRINT FAILURE • REPEATING REGRESSION IS A BIG TASK AND AN OVERHEAD • MICROSOFT HAS NO TESTERS NOT EVEN GOOGLE, FACEBOOK AND CISCO • 15K+ DEVELOPERS /4K+ PROJECTS UNDER ACTIVE • IN A “MOBILE-FIRST AND CLOUD-FIRST WORLD.” DEVELOPMENT/50% CODE CHANGES PER MONTH. • THE EFFORT, KNOWN AS AGILE SOFTWARE DEVELOPMENT, • 5500+ SUBMISSION PER DAY ON AVERAGE IS DESIGNED TO LOWER COSTS AND HONE OPERATIONS AS THE COMPANY FOCUSES ON BUILDING CLOUD AND • 20+ SUSTAINED CODE CHANGES/MIN WITH 60+PEAKS MOBILE SOFTWARE, SAY ANALYSTS • 75+ MILLION TEST CASES RUN PER DAY. • MR. NADELLA TOLD BLOOMBERG THAT IT MAKES MORE • DEVELOPERS OWN TESTING AND DEVELOPERS OWN SENSE TO HAVE DEVELOPERS TEST & FIX BUGS INSTEAD OF QUALITY. SEPARATE TEAM OF TESTERS TO BUILD CLOUD SOFTWARE. • GOOGLE HAVE PEOPLE WHO COULD CODE AND WANTED • SUCH AN APPROACH, A DEPARTURE FROM THE TO APPLY THAT SKILL TO THE DEVELOPMENT OF TOOLS, COMPANY’S TRADITIONAL PRACTICE OF DIVIDING INFRASTRUCTURE, AND TEST AUTOMATION.
  • Accelerate Software Innovation Through Continuous Quality

    Accelerate Software Innovation Through Continuous Quality

    Accelerate Software Innovation Through Continuous Quality 1 Software quality is recognized as the #1 issue IT executives are trying to mitigate. Enterprise organizations strive to accelerate the delivery of a compelling user experience to their customers in order to drive revenue. Software quality is recognized as the #1 issue IT executives are trying to mitigate. QA teams know they have issues and are actively looking for solutions to save time, increase quality, improve security, and more. The most notable difficulties are in identifying the right areas to test, the availability of flexible and reliable test environments and test data, and the realization of benefits from automation. You may be facing many challenges with delivering software to meet the high expectations for quality, cost, and schedule driven by the business. An effective software testing strategy can address these issues. If you’re looking to improve your software quality while achieving your business goals, Parasoft can help. With over 30 years of making testing easier for our customers, we have the innovation you need and the experience you trust. Our extensive continuous quality suite spans every testing need and enables you to reach new heights. 3 QUALITY-FIRST APPROACH You can’t test quality into an application at the end of the software development life cycle (SDLC). You need to ensure that your software development process and practices put a priority on quality- driven development and integrate a comprehensive testing strategy to verify that the application’s functionality meets the requirements. Shift testing left to the start of your development process to bring quality to the forefront.