DOCSLIB.ORG

Amplifies Regression Testing DOLBY LABORATORIES

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Amplifies Regression Testing DOLBY LABORATORIES This article appears here with permission from the publisher of SAPinsider, WIS Publishing. DOLBY LABORATORIES Amplifies Regression Testing Sound Pioneer Finds “Peace of Mind” with Risk-Based Approach by Ken Murphy, Editorial Director At a Glance Dolby Laboratories, a pioneer in advancing the sci- ence of sight and sound, would not be synonymous with delivering the best entertainment experience Goal: Reduce or eliminate without a strict adherence to quality assurance. If regression test coverage gaps and you’re enjoying the latest blockbuster in a cinema lengthy testing integration cycles theater, watching a binge-worthy show on a televi- with a risk-based automated testing sion, or even listening to a conference call at home solution or on your smartphone, Dolby likely had a hand in Strategy: Integrate SAP ERP, SAP the technology behind it to make it look and sound SuccessFactors applications, and great. Dolby technologies have made it the leader in other technologies with the Tricentis creating entertainment experiences that reach bil- Tosca Continuous Testing platform lions of people worldwide through partnerships with Outcome: End-to-end regression storytellers, businesses, and consumers. test automation covering In 2016, Dolby’s Enterprise Applications area approximately 40% of mission- critical applications (50% sought an equally stringent level of quality assurance target by end of fiscal year); for its business process testing when inconsistent roughly 75% savings in hours on testing coverage and a lack of testing standardization regression testing; and a risk-based revealed risk coverage gaps and lengthy integration prioritization and documentation of cycles that had the potential to introduce risk to regression testing library according mission-critical processes, including its customer- to business risk objectives facing Dolby Reporting Portal, which its licensees use to report royalties. SAPinsiderOnline.com | Issue 2 • 2018 “In 2016, we put forward our vision and mission state- ments to include continuous quality assurance by devel- oping, documenting, and maintaining a comprehensive change management program to include test automation,” says Mamatha Mitr, Senior IT Manager of Dolby’s Enter- prise Applications. Mitr heads up the strategy for Testing Center of Excellence (TCoE), a byproduct of Dolby’s efforts to introduce standards to its testing landscape. A focus area for Dolby was to provide end-to-end cover- age for what Dolby refers to as its financially significant (FINSIG) applications, to include several SAP ERP Finan- cials applications (for order-to-cash, sales and distribution, and procure-to-pay processes) as well as SAP SuccessFactors Employee Central. These are all key test cases for Dolby’s Sarbanes-Oxley audit compliance. “When we sought and selected a tool for automated test- Dolby Laboratories ing, we wanted it to lend itself to the different technologies in our landscape, which includes SAP software,” Mitr says. Headquarters: San Francisco, California “We have a large SAP footprint in terms of our FINSIG Industry: Entertainment and communications applications.” (audio, video, and voice technology) Revenue: $1.081 billion (2017) Gap Compression with a Risk-Based Approach One of Dolby’s requirements was that an automated test- Employees: 2,122 (September 2017) ing tool follow a risk-based approach, which, in addition to Company details: diminishing risk through early and end-to-end testing auto- • Founded by Ray Dolby (1933-2013) in 1965 mation, also estimates the potential damage that can occur • Brand anthem: On a quest for the spectacular based on requirement levels for the affected software. • The Dolby Theatre has hosted the Academy With a risk-based approach as a key requirement, and Awards ceremonies since 2002; Ray Dolby based in part on a Gartner assessment, Dolby opted to received a star on the Hollywood Walk of Fame partner with Tricentis. The Tricentis Tosca risk-based plat- form helps to optimize test cases with its Linear Expansion directly in front of the Dolby Theatre methodology that reduces the number of test cases while • Dolby Laboratories has received 10 Academy optimizing risk coverage. The software enables customers to Awards and 14 Emmy Awards assess an aggregated risk coverage from its automated test- • Dolby has over 140 Dolby Cinema sites open ing requirements. around the world “The reason Dolby selected this solution was because it is • Netflix, Amazon, Disney, Tencent, Apple, Warner completely risk-based,” Mitr explains. “Its two key attributes Bros., Samsung, Sony, and more have chosen are frequency and damage class. Frequency is a measure of Dolby Vision HDR and Dolby Atmos immersive how often the associated item (requirement, user story, etc.) audio to deliver entertainment experiences is used and the damage class is the potential impact to the SAP solutions: business — the risk or damage that could result from the SAP ERP Financials, SAP failure of the associated item.” SuccessFactors Employee Central, SAP Supply Chain In June 2016, Dolby began a pilot program by selecting Management, SAP Fiori, SAP Solution Manager, test cases that spanned various applications and work- SAP Concur solutions, SAP governance, risk, and streams. The scripts for customer-facing applications such as compliance solutions, SAP Analytics solutions, and Dolby Reporting Portal were in the mix. After the 90-day SAP Business Warehouse powered by SAP HANA pilot program, Mitr and her team categorized the benefits into two buckets, tangible and intangible, to determine re- turn on investment (ROI). Metrics for the former included hours saved per testing cycle, number of reported defects SAPinsiderOnline.com | Issue 2 • 2018 “One of the greatest intangible benefits was when we surveyed everyone who worked with us in terms of automation or technical tasks, we heard consistent feedback that they now had peace of mind with automated testing.” — Mamatha Mitr, Senior IT Manager, Enterprise Applications, Dolby Laboratories resulting from testing automation, and the percentage of “Surround” Coverage automation coverage versus the coverage during manual In September 2016, Dolby began a deep dive into test cycles. “That was an important metric,” says Mitr. Tricentis Tosca. Before expanding its regression test case “When testing was manual, our business system analysts library, however, the company wanted to establish a firm weren’t running all of the tests because of limited time footing with a regression test bed feathered with docu- and resources. They only ran the portion that touched mentation for the test cases run during the pilot. With the code, and that prevented us from having end-to-end the Dolby Reporting Portal, for example, the TCoE team test coverage.” wanted to verify that all the test steps were covered. This This scenario echoes the traditional challenges facing was an important task, according to Mitr, because the test an enterprise that tries to mirror a risk-based approach team did not necessarily have the functional experience with limited resources; the workaround to mimic risk- to be well versed in the application. based automation is either to test exhaustively or to test “First and foremost, the TCoE team got a functional at random. By prioritizing test coverage based on fre- walkthrough of the technology and the application,” says quency and potential damage, the Tricentis Tosca Risk Mitr. “We had a requirements-gathering session to ensure Optimization tool assigns an absolute weight in its Linear we understood the documentation. From there, we start- Expansion algorithm to continuously ensure that testing ed a sprint planning exercise coupling it with a detailed activities are aligned with business risk objectives. charter and a scope deck.” Mitr says that running test case scripts for customer- With the planning exercise, the team matched its busi- facing applications during the pilot led to a consistent ness risk objectives with the Tricentis Tosca Risk Optimi- drop in the number of incidents, measured by a reduc- zation tool based on frequency and damage class. A re- tion in tickets, through three monthly release cycles. The gression test bed and robust documentation set allowed growing TCoE team, which started with a handful of full- Dolby to then branch out into more of an agile method- time employees, also proved ROI with significant savings ology. “Once we went through a few release cycles, we in hours per testing cycle. measured ourselves to make sure that we met our success As for the intangible benefits resulting from the pilot metrics, and then we moved on to the next workstream,” program, Mitr says that Dolby quickly developed a stan- Mitr says. “We had a varied methodology — agile, water- dardized approach for all of its testing deliverables. “We fall, and hybrid — and we catered our planning accord- came up with a robust maintenance process,” she says. ingly to match the methodology of the workstream.” “Subsequent to doing automation for a particular work- stream, we were able to document all of that for ongo- Crystal-Clear Workstreams ing maintenance. One of the greatest intangible benefits By September 2018 — the close of Dolby’s fiscal year — was when we surveyed everyone who worked with us in Mitr says the goal is to have automated regression testing terms of automation or technical tasks, including func- for at least 50% of its FINSIG applications. As of April, tional consultants and system managers, we heard con- the TCoE team had already reached the 40% mark. “I es- sistent feedback that they now had peace of mind with timate that we’ll exceed our target and be at roughly 65% automated testing.” by the end of the fiscal year,” she says. SAPinsiderOnline.com | Issue 2 • 2018 How Tricentis Helped Dolby Laboratories Reduce Its SAP Testing Burden With an industry-leading platform for continuous testing, Tricentis is recognized Dolby has over 140 Dolby Cinema sites open for re-inventing software testing for around the world DevOps. Through risk-based testing, scriptless end-to-end test automation, and extensive SAP technology support, Tricentis breaks through the barriers experienced with conventional SAP software testing methods.
Load more

Recommended publications
  • Tricentis Tosca Mobile Testing
    DATASHEET Tricentis Tosca Mobile Testing Overview Key Benefits Today’s enterprises are developing and deploying new “customer Mobile testing without programming experience” enhancements via mobile apps at an unprecedented Tricentis Tosca lets you create mobile tests in the same way rate. How can you ensure that each mobile interface is robust that you create UI and API tests—just drag and drop. You gain and seamlessly interacts with the functionality provided by your the power of Appium, the industry-standard mobile testing tool, core applications—especially when “mobile” transactions span without having to deal with Appium’s technical complexity. mobile, APIs, and other components that might all be evolving No complex setup or scripting is required. at different paces? Fully integrated into Tricentis Tosca Tricentis Tosca’s Mobile Engine 3.0 is designed to help testers There’s no need to have a disparate assortment of standalone integrate mobile testing into their broader testing strategy. By tools for testing each part of your enterprise system landscape. pairing the power of Appium with Tricentis Tosca’s Modelbased Tricentis Tosca provides you with a single platform that tests your Test Automation, we help you rapidly create mobile tests and use native, hybrid, and web applications for iOS and Android inde- them as building blocks for end-to-end test scenarios that might pendently and as part of multi-channel (e.g., mobile + API + SAP…) involve mobile, APIs, SAP, and other enterprise technologies. end-to-end test scenarios. You can start running test cases by first simulating data on your mobile device, followed by checking the data in your back-end system using a web service or database interface.
    [Show full text]
  • SAP S/4HANA Migration Guide Key Steps for Faster, Safer SAP Updates
    SAP S/4HANA Migration Guide Key Steps for Faster, Safer SAP Updates Tricentis | SAP S/4HANA Migration Guide | 1 SAP customers face a major challenge when migrating their businesses to S/4HANA. While upgrading to S/4HANA promises many benefits, it is not without risk. Not only will the migration impact SAP applications, but also the customizations and connected technologies (UIs, APIs, mobile, databases, third-party applications, etc.) that collectively run your business. Poorly planned migrations can lead to cost overruns, system downtimes, security lapses, and other serious business consequences. And the migration is only the beginning. With S/4HANA, SAP is promising more frequent SAP updates than ever before. To prepare for these updates, organizations need to be confident that their development, testing, and training processes can handle the speed and scope required to implement these updates safely, without risk to the business. Whether your migration approach is a new implementation (greenfield approach) or system conversion (brownfield or bluefield approach), this guide will show you how to gain certainty in your roadmap to S/4HANA and avoid false starts and unexpected costs along the way. In this guide, we’ll cover the 4 crucial steps to a successful S/4HANA migration: 1. Get a custom S/4HANA migration assessment to identify the technical and business risks involved. 2. Identify the testing needed to address those risks. 3. Optimize your test suite for resilient, risk-based testing. 4. Use a continuous testing platform to automate and accelerate ongoing testing. 1. Get a Custom S/4HANA Migration Assessment Like a crystal ball, a S/4HANA migration assessment lets you see into the future.
    [Show full text]
  • Enterprise Test Automation Speed Changes Everything
    Enterprise Test Automation Speed changes everything. Today’s enterprises are undertaking digital trans- ly driven by AI. It addresses both agile develop- formation to boost speed, agility, and innovation, ment and complex enterprise apps—increasing while keeping an eye on costs. Yet, traditional soft- software delivery speed, reducing costs, and im- ware testing hasn’t evolved with the new world of proving quality. Now, testing takes just minutes Agile and DevOps—not to mention the complexity or hours instead of days or weeks, while being far of large, enterprise apps— and has become the more cost effective and highly accurate. This kind #1 barrier to software release speed and agility. of enterprise automation drives massive efficiency Enter Tricentis. We offer a new and fundamentally and speed gains. And the potential of digital trans- different way to tackle software testing, dramati- formation becomes abundantly clear. cally accelerating software delivery. Our approach is totally automated, fully codeless, and intelligent- Platform Benefits ACCELERATE RELEASE SPEED REDUCE TESTING COSTS INCREASE SOFTWARE QUALITY Leveraging artificial intelligence Our platform is AI-driven and Boost release confidence and and the efficiencies of the codeless, helping you eliminate obtain business risk coverage cloud, you can accelerate soft- error-prone manual testing >85%. The quality of your soft- ware delivery by reducing test and achieve end-to-end test ware goes up. The stress of cycle times as much as 80%. automation rates >90% to re- your day goes down. duce costs. >80 >90 >85 %FASTER %TEST AUTOMATION %RISK COVERAGE © 2021 Tricentis USA Corp. All rights reserved 1 The World’s #1 Testing and Automation Platform ADVANCED ANALYTICS ro Proect and Portfolio iiilit ontino etin and Atomation Platform Anale anae Atomate Assess Risk Design and Build Execute at Scale Reduce testing efforts by exposing risks Design manual, open source and Scale ditrited endtoend tetin in critical business processes nocode atomated tet from mainframe to moile A.I.
    [Show full text]
  • Software Testing Reinvented for Agile and Devops
    Tricentis: Software Testing Reinvented for Agile and DevOps Tricentis accelerates software testing for digital transformation. Our industry-leading Continuous Testing Platform provides auto- mated insight into the business risks of your software releases—transforming testing from a roadblock to a catalyst for innovation. #1 Continuous Testing Platform— Accelerate Cloud, SAP, Agile, and DevOps Initiatives Agile Automated Distributed Dev Testing Continuous Testing Load Testing Open source testing and agile test Resilient regression testing at the Cloud-based performance testing management speed of change labs on-demand Tricentis Analytics Cross Project and Portfolio Visibility Benefits Expose change impacts in minutes with advanced, resil- Accelerate release cycles by orchestrating and scaling ient test automation optimized for 150+ technologies testing efforts across your teams, projects, applications, Modernize testing across SAP and packaged apps and tools (including open source) with the most comprehensive testing solution for the Reduce risks with centralized visibility/traceability, intelligent enterprise predictive analytics, and “release readiness” dashboards Results >90% 5X >80% Test Automation Testing Speed Increase Risk Coverage www.tricentis.com © 2019 Tricentis GmbH. All Rights Reserved In the News Sample Customers DEC 3, 2018 Tricentis Named a Leader in the 2018 Gartner Magic Quadrant for Software Test Automation for Fourth Year in a Row NOV 18, 2018 The 18 Hottest Companies in Enterprise DevOps JUNE 20, 2018 Recognized as
    [Show full text]
  • Modernizing SAP Testing in the Energy Industry
    Modernizing SAP Testing in the Energy Industry Tricentis | Modernizing Software Testing in the Energy Industry | 1 Despite the growing emphasis on digital transformation, the average test automation rate remains a dismal 18% overall— and 6% for energy companies, who are commonly working with complex systems including SAP and custom applications. This means that each time the SAP infrastructure or business processes change, enterprises rely on internal business users or dedicated manual testers to click through each critical business process and note whether the expected results are achieved at every step. As you can imagine, this is an incredibly slow and costly process: • Completing a single test cycle commonly takes weeks or even months • Testing is routinely cited as the #1 bottleneck to delivery speed • Testing consumes 30-40% of the average IT budget Now , energy companies are increasingly turning to SAP innovations such as SAP S/4HANA and SAP Cloud to optimize production, extend business processes to capitalize on new opportunities, and accelerate innovation across the value chain. As a result, the pace of change and the complexity of what needs to be tested are constantly increasing dramatically. This presents a number of challenges from a testing perspective: • Automated tests architected for previous generations of SAP platforms must be rebuilt. • Existing testing processes don’t suit the new faster (and more fragmented) release cycles, with different components evolving at different speeds. • Within the SAP ecosystem, testing must address SaaS cloud solutions such as SAP SuccessFactors, SAP Ariba, SAP Fieldglass, and SAP Hybris. • Reinvented business processes commonly extend beyond the SAP ecosystem, including APIs, mobile, mainframes, and so on—but legacy testing approaches often do not.
    [Show full text]
  • Continuous Load Testing: Reinventing Load Testing for Devops
    Continuous Load Testing: Reinventing Load Testing for DevOps tricentis.com Executive Overview Transformational initiatives such as Agile and DevOps are compelling software development teams to change their approach to load testing. This new approach involves extending beyond late-cycle load testing by performance testing specialists and empowering developers and testers to perform early, continuous load testing. Flood IO is built to enable load testing for DevOps, helping DevTest teams gain performance feedback rapidly and continuously. In addition to supporting industry-leading load testing tools, Flood integrates with Tricentis Tosca—allowing users to perform load testing with the same scriptless tests they use for functional testing. For DevOps, Load Testing Shifts Left Over 91% of organizations already have Agile or DevOps initiatives or plan to adopt them within the immediate future. As part of the process transformation required to adopt these more iterative and rapid methods of delivering software, organizations need to reassess ALL practices associated with software development and test—including load and performance testing. Traditionally, load testing has been deferred until the late phases of waterfall release cycles. Specialized performance testers applied legacy load testing tools to validate application performance prior to release. Why is this no longer sufficient? • With teams moving to more componentized applications (including cloud-native apps, microservices, etc.), an application involves many highly-distributed components—and
    [Show full text]
  • Integration Overview How Tricentis Qtest and Neoload Can Provide a Unified View of Application Quality
    INTEGRATION OVERVIEW HOW TRICENTIS QTEST AND NEOLOAD CAN PROVIDE A UNIFIED VIEW OF APPLICATION QUALITY These days, more QA/Testing and Developer teams are opening up their testing process to stakeholders across the organization by tracking functional requirements, manual and automated test cases, testing coverage, and other vital elements and metrics of the testing process. Having all this data available in one solution allows all business stakeholders, including product owners to have a single, unified vision of their quality systems. Performance Management is Different Performance management always stands apart and get managed differently than other business-critical processes - including the way the results of performance testing are shared. Sharing performance reports have traditionally been done by exchanging documents, emails, presentations, or by editing an Excel spreadsheet. Today’s best practices for performance testing include exposing all performance data to developers with the help of CI/CD plugins. Organizations that have adopted Agile workflows and processes are finding that moving towards adoption of a DevOps approach is becoming a mandatory step if they want to get the most benefit for stakeholders across the business - Developers, QA managers, and products owners - from their performance testing insight. Value for Business Stakeholders Non-technical managers and business executives, who are not performance testing practitioners, do not need to understand all the in-depth technical details of performance testing to make informed decisions or efficiently manage performance testing projects. But, they still need some level of deep insight into the performance of applications to make directional, investment and resource decisions. This need makes it a business imperative to expose performance testing results and KPI to managers and other non-testing stakeholders across the business, directly from Test Management solutions such as qTest.
    [Show full text]
  • Advanced End-To-End SAP on AWS Testing with Tricentis Tosca
    Get Started Advanced End-to-End SAP on AWS Testing with Tricentis Tosca Optimize you SAP investments with the Gartner Top Rated Continuous Testing Platform Tricentis is an AWS Advanced DevOps Competency Partner. Industry trends & challenges SAP S/4HANA on AWS is fast becoming the centerpiece of enterprise Digital Transformation initiatives. However, conventional and legacy software testing approaches lack comprehensive capabilities to fully test SAP across all touch points in complex landscapes. Tricentis Tosca, the Gartner top Rated Continuous Testing Platform for SAP, helps leading enterprises modernize testing for SAP Digital Transformation initiatives. Tricentis Tosca for SAP Tricentis has changed the game with a new approach to SAP business process testing. Tricentis has established an advanced SAP business process test automation technology that lets you effortlessly prevent defects to your core end-to-end business transactions - across and beyond SAP (APIs, web UIs, Java, mobile interfaces, mainframes, data warehouses, and more). Benefits Enterprises who need to accelerate testing for SAP rely on Tricentis. Our industry-leading Continuous Testing Platform provides automated insight into the business risks associated with every release. Using Tosca, we have seen that customers can test 10x faster with visibility into business risks. Automated End-To-End Testing for SAP Load Testing for SAP Fiori Automatically test your SAP implementation Easily construct and run load tests and beyond. Test business processes across specifically tuned to support SAP Fiori all connected applications + APIs Validate SAP Workloads on AWS AI-Enabled Testing for the Business User Prevent business disruption by revalidating Tricentis Tosca’s model-based test SAP business processes – measure automation approach enables business performance impacts across each critical users to rapidly create resilient, reusable path.
    [Show full text]
  • Performance Engineering a Sogeti and Neotys Report the State of Performance Engineering — Contents
    The State of Performance Engineering A Sogeti and Neotys report The state of performance engineering — contents Introduction to performance engineering 3 Chapter 1: Culture 6 Enterprise spotlight: IMA 15 Chapter 2: Organisation 18 Chapter 3: Practices 27 Enterprise spotlight: MAIF 42 Chapter 4: Building in performance engineering 44 Chapter 5: Outlook 57 Closing thoughts: the practitioners’ corner 68 Acknowledgements, About the study, Respondents, Footnotes and references, About Neotys, About Sogeti 74 2 THE STATE OF PERFORMANCE ENGINEERING 2020 PRESENTED BY SOGETI AND NEOTYS Introduction In the first half of 2020, application performance became Welcome to the State of Performance highly visible and imperative. At the time of writing this Engineering report report, video conferencing, teleconsultation, online shopping, and many online transactions were all in the The discipline of performance engineering is often midst of extraordinary surges of use. Companies guided reserved for specialists. There is little data available by a strong proactive vision in digital business have to enterprises to help them understand the current managed to navigate successfully in these uncertain practices and how other organisations are managing it. external circumstances. For instance, UK home furnishing This research fills the void by combining the opinions retailer Dunelm reported a 100%+ increase in sales of 515 senior decision makers and the perspective during the lockdown weeks of Spring 2020. of subject matter experts from Sogeti, Neotys and Organisations are changing the way they operate — outside practitioners to explore the landscape of with a surge in remote work — and they increasingly performance engineering. This report reveals the place serve their customers through digital engagement.
    [Show full text]
  • Performance Testing for the Modern Enterprise Executive Summary
    LoadRunner vs. NeoLoad: Why Change? Performance Testing for the Modern Enterprise Executive Summary Enterprises are increasingly finding that their performance testing has not kept pace with advances in software development. Development cycles are getting shorter, with new applications, microservices, APIs and features being released faster and faster. But all too often performance testing remains a highly manual, time-consuming undertaking that requires expertise and specialized know-how. Performance testing has become a bottleneck. That’s why organizations are looking for a more modern and productive alternative that breaks down the silos of expertise, clears the QA logjam and gives every team — experts and non-experts alike — the ability to test faster, cost-effectively, across a broad range of enterprise needs. Because how do you release fast when QA is stuck? And get unstuck when that requires deep expertise not shared by the organization at large? NeoLoad accommodates different types of testers across the entire organization. Everyone from centralized performance engineering experts doing complex end- to-end testing in a traditional approach to autonomous users testing individual components and APIs in a DevOps environment. They use one solution to test monolithic packaged apps like SAP or Oracle, and to automate API tests. With NeoLoad, your teams require less time to do more work, more accurately, with fewer people. You can have your experts focus on things with higher strategic value instead of the “grunt work” of perpetually maintaining and rewriting scripts every time code changes. Enterprise Performance Testing for Today’s Reality Enterprises are finding that the performance testing practices, methodologies and tools that have served them well for the past 10–15 years are no longer sufficient to meet the pace of today’s faster release cycles.
    [Show full text]
  • The Evolution of Test Automation Results from the 2018 Tricentis and Techwell Survey
    The Evolution of Test Automation Results from the 2018 Tricentis and TechWell Survey BROUGHT TO YOU BY: The Evolution of Test Automation Table of Contents Introduction 3 Current Automation Levels and Future Outlook 4 Which Tests Are Being Automated, and by Whom? 6 Benefits and Challenges 8 Test Automation Was Born out of Necessity 10 Test Automation Tools and Frameworks 12 Conclusion 14 2 The Evolution of Test Automation Introduction ver the past few years, agile and DevOps Tricentis and TechWell Corporation partnered have moved from buzzwords to common on a survey to measure how widespread test practice. So much so, in fact, that if you haven’t automation is today and gather data on how already implemented agile or DevOps, you are much those numbers are expected to change lagging behind. With the increasing popularity over the next few years. The survey also looks of these collaborative, fast-moving, adaptable at how test automation is being integrated into principles, the software development and testing teams; the benefits and challenges organizations lifecycle has become lightning-fast. Because this face; the relationship between agile, DevOps, lifecycle is so fast, test automation is mandatory to and test automation adoption; and the tools and maintain the quality of your software. If you aren’t frameworks organizations are utilizing for their automating testing, you’re probably losing sleep automation efforts. worrying that something catastrophic is going to happen and you won’t even know about it. The data collected from 173 software testers, developers, QA decision-makers, and business Organizations of every size now recognize the users was analyzed with an eye toward how need to speed up their processes if they want to enterprise-level companies compare to smaller stay competitive in the marketplace.
    [Show full text]
  • A Practical Guide to Continuous Performance Testing Table of Contents
    A practical guide to continuous performance testing Table of contents 1. Introduction 3 2. Getting started 4 a. Assess your gaps 4 b. Prioritize, then systematize 5 c. Prioritization methodologies 6 3. Lay the foundation for success 7 4. Start with the end in mind 8 a. Distinguish between SLA, SLO, and SLI 8 5. Pick the right targets to automate 9 a. Pick the right teams 10 b. Why APIs are easier 10 c. How early is early enough to performance test APIs? 11 6. Make scripting easy for multiple teams 12 7. Co-locate app and test automation code 13 8. Develop performance pipelines 14 9. Use dynamic infrastructure for test environments 15 10. Ensure trustworthy go/no-go decisions 18 a. Reduce cognitive thrash 19 b. Layer in guardrails 20 11. Conclusion 20 12. About Tricentis 21 INTRODUCTION This guide presents pragmatic considerations and experienced-based best practices for performance engineers looking to automate performance testing in CI/CD pipelines. The #1 challenge we hear from those responsible for performance testing is the difficulty in keeping up with the pace and scale of development teams so that you can deliver performance feedback early and often. They’re caught in a crossfire between performance being more imperative than ever — slow or buggy systems and applications have a material impact on customers/users, brand reputation, and bottom-line revenue — and releases coming faster than ever. The clock speed of testing is mismatched to the speed of development. With today’s automated builds taking minutes — not hours (or days) — integrating continuous performance testing into automated pipelines is the only answer.
    [Show full text]