DOCSLIB.ORG

Resilience Testing

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Resilience Testing Workshop: Resilience Testing Geoffrey Arij van der Tas Mark Abrahams About us! About Today • Setup our lab environment • Theory: • What is Resilience? • Resilience • How to test Resilience? • Load Testing • Resilience Testing • Automated Resilience Testing What is Resilience? “ Resilience is the ability of a system to withstand a major disruption within acceptable degradation parameters and to recover within an acceptable time and composite costs and risks. ” Becoming Resilient Examples: Load Balancing Stand-by servers Infra Examples: People & Re-try Pattern Software Examples: Processes Circuit Breaker Pattern Yearly Datacenter Switch-over Stand-by shifts To become Resilient • Prevent: Measures you take to make sure problems do not occur • Resist: To stand firm, to withstand problems that occur • Remediate: Is to correct or make the service again right Prevent • Recover: Restore the service as quickly as possible Recover Resist Remediate Now it is up to you.. Our App VM Front End API DB Installing Lab environment from HDD • Install Virtualbox • Install JDK • Install Putty or other ssh client • Copy Gatling • Install Notepad ++ or an IDE/other Scala editor • Make sure you set your JAVA_HOME to the JDK Setup carshare VM server • Get the .ova file from USB drive • Import the .ova file by clicking or via Virtualbox -> file -> import appliance • Make sure network connection is set to bridged mode: • Settings -> network -> network -> from dropdown choose “Bridged Adapter” • Start vm and login with user “root” and password “root” • Run command “ifconfig” and get ip from “eth0” interface • Run command “start-carshare” • Navigate in browser to url “http://{{OBTAINED_IP}}:8080” • Connect to the Virtual Box via Putty {{OBTAINED_IP}}:22 Load Testing “Load testing is the process of putting demand on a system and measuring its response” Most Populair Tools: JMeter, Gatling, Neoload & Loadrunner . We are going to use: Gatling. Forms of Performance Testing • Load Testing • Spike Testing • Stress Testing • Endurance Testing Load testing Baseline scripts: Load test Stress test VM Front End API DB Load Test Script: carshareBrowse.Load Users: 20 Runtime: 5 minutes Go to Gatling\user-files\simulations\carshare Open the file called load (with Notepad ++ or a IDE (with scala plugin)) Change .baseUrl (“http://OBTAINED_IP:8080”) Stress Test Script: carshareBrowse.Stress Runtime: 5 Minutes Users: 1000+ Go to Gatling\user-files\simulations\carshare Open the file called stress(with Notepad ++ or a IDE (with scala plugin)) Change .baseUrl (“http://OBTAINED_IP:8080”) Resilience Testing “Resilience testing measures the ability to absorb the impact of a problem in one or more parts of a system.” Tools: Stress, Nstress Resilience testing Scripts: Load test VM Front End API DB CPU Spike on VM # Stress CPU with one worker process for 20 seconds stress –c 1 –t 20 Memory Spike on VM # Stress memory for 20 seconds stress –m 5 -t 20 IO Spike on VM # Stress io for 60 seconds stress –i 1 –t 60 Network Failure on VM #Short network outage /etc/init.d/networking restart Automated Resilience Testing “Resilience testing measures the ability to absorb the impact of a problem in one or more parts of a system.” Most Populair Tools: Chaos monkey, Gremlin. We are going to use: Chaos monkey for kubernetes. Test results – normal load, normal circumstances Test app as monolith Application Test app as microservice (non HA) Test results – normal load, pod failure chaos Test app as microservice (HA) Test results – normal load, pod failures chaos HA Resilience testing Why: To help you build more stable application and perform well in real life situations; To avoid: Service loss, Data loss, Customer loss! How: By creating chaos, testing ‘What if scenarios’ & investigate what happens; What: Be creative and use the following tools for example: Tools you can use Load/Stress tool Resilience Stress tool - Nstress - Chaos monkey - Stress - Gremlin Application Load Monitoring - Gatling - ELK stack - Loadrunner - Dynatrace - Jmeter - Prometheus - NeoLoad Credentials: Geoffrey van der Tas Mark Abrahams https://www.linkedin.com/in/geoffreyvdtas/ https://www.linkedin.com/in/mark-abrahams- b8218129/ [email protected] @gavdtas [email protected] http://geoffreyvdtas.com Want to read more • https://gatling.io/ • https://usersnap.com/blog/resilience-testing/ • https://www.ibm.com/cloud/blog/resilience-testing-insights-from-the-pros • https://medium.com/netflix-techblog/fit-failure-injection-testing- 35d8e2a9bb2 • https://medium.com/netflix-techblog • https://www.gremlin.com/community/tutorials/chaos-engineering-the- history-principles-and-practice/ • http://geoffreyvdtas.com/blog (a blog with more information will follow soon) or https://drive.google.com/open?id=19UC8m9eLyrViGLdkg7hn70IngIoqIWO1 - to download the stuff from the workshop.
Load more

Recommended publications
  • Kickstart Your Gatling Performance Testing
    Kickstart Your Gatling Performance Testing Siegfried Goeschl Version 1.0.0, 2018-11-04 Introduction 1 Siegfried Goeschl • Senior Software Engineer • Writing server-side code • Java Meetup Vienna co-organizer • Apache Software Foundation member • Currently working at Erste Bank Austria 2 Introducing Gatling • Performance testing framework • Tests are written in in Scala • Developer-centric test tool • Development started in 2010 • Gatling 3.0.1 released now • Since V3 there are two license models - free & commercial. • As you know some guys have a strong opinion about OSS 3 What Linus Says 4 5 • Having said that a commercial license could generate more revenue keeping the Open Source version alive. 6 Money Makes The World Go Round 7 Gatling vs. FrontLine • Gatling Open Source is under ASL 2.0 • Gatling FrontLine is the enterprise edition ◦ Annual license or "pay as you go" ◦ Web-based, ◦ More bells & whistle ◦ Real-time reporting 8 Under The Hood • Supports HTTP 1.1/2.0 & JMS protocol • Response validation ◦ Regular expressions ◦ XPath & JSONPath ◦ CSS selectors 9 Under The Hood • Provides Domain Specific Language (DSL) • Uses asynchronous non-blocking HTTP client • Integrates with Maven, SBT & Gradle • Test data feeders CSV, JSON, JDBC, Redis • Management-friendly HTML reports • No more 1:1 mapping between virtual users and worker threads. 10 When To Use Gatling? • Want to write test code in your IDE? • Need some integration & performance tests? • Want to run those test on your CI server? • Do you care about reviews and version control? 11 Getting Started 12 Getting Started • JDK 1.8 • Apache Maven 3.5.x • IntelliJ Community Edition • IntelliJ Scala Plugin 13 Getting Started • https://github.com/gatling/gatling-maven-plugin-demo • Import the Maven project into your IDE • Write and debug Scala code there • Execute Gatling tests on the command line • Simple CI integration using Maven • The official Gatling distributable is not suited for development.
    [Show full text]
  • Tero Suominen PERFORMANCE TESTING REST APIS Information
    Tero Suominen PERFORMANCE TESTING REST APIS Information Technology 2017 PERFORMANCE TESTING REST APIS Suominen, Tero Satakunta University of Applied Sciences Degree Programme in Computer Science December 2017 Number of pages: 39 Appendices: 0 Keywords: REST, performance testing, automation ____________________________________________________________________ The subject for this thesis was performance testing REST APIs that had been imple- mented into a Java application. The purpose of this research was to come up with a method on how the performance and functionality of the REST APIs could be meas- ured and tested within Profit Software. The research consisted of two parts. First, I searched to find an existing software ca- pable of being used for testing REST APIs. After selecting the tool that would be used to create the performance tests, a local test environment was set up that allowed us to estimate the capability of the software and the method of testing itself. The local envi- ronment consisted of the same components and software that could be used also in the already existing test environments within the company. This way moving the tests from the local environment into the actual test environment went smoothly. With the help of this research we were able to detect issues with the functionality of some APIs, when they were under load. We were able to fix these issues in the imple- mentation during the development phase and after changing the implementation we could verify that the APIs functioned correctly by using these same tests. REST RAJAPINTOJEN SUORITUSKYKYTESTAUS Suominen, Tero Satakunnan ammattikorkeakoulu Tietotekniikan koulutusohjelma Joulukuu 2017 Sivumäärä: 39 Liitteitä: 0 Asiasanat: REST, suorituskykytestaus, automaatio ____________________________________________________________________ Opinnäytetyön aiheena oli Java-sovellukseen toteutettujen REST rajapintojen performanssitestaus.
    [Show full text]
  • Loadrunner Professional and Loadrunner Enterprise 2021
    Product Flyer Application Delivery Management LoadRunner Professional and LoadRunner Enterprise 2021 License Bundles Community bundles and license bundles available for LoadRunner Professional and LoadRunner Enterprise 2021. Learn more at Community Bundles (Free of Charge) https://software.microfocus.com/ Bundle Name Description products Community 50 Vusers for all protocols, except for Templates and GUI Mobile UI Unlimited Vusers for the TruClient—Native Mobile protocol. Note: This bundle requires an Micro Focus Mobile Center license. For more details, see: https://software.microfocus.com/software/customer-technical-support-services License Bundles Bundle Name Protocols .NET Record/Replay Microsoft .NET Database ODBC Oracle—2 Tier Development SDK Unit Test—NUnit, Junit Selenium, LeanFT* GUI Unified Functional Testing (UFT) Java Java Record Replay Java over HTTP Mobile and IoT DevWeb JMeter Gatling Web—HTTP/HTML TruClient—Mobile Web SMP (SAP Mobile Platform) MQTT (Internet of Things) CoAP (Internet of Things) Network DNS (Domain Name Resolution) FTP (File Transfer Protocol) IMAP (Internet Message Access Protocol) LDAP (Listing Directory Service) MAPI (Microsoft Exchange) POP3 (Post Office Protocol) SMTP (Simple Mail Transfer Protocol) Tuxedo Windows Sockets (Winsock) Oracle E-Business DevWeb JMeter Gatling Oracle NCA Oracle—Web Siebel—Web Web—HTTP/HTML * Continued on next page LeanFT support is for LoadRunner Enterprise only. Contact us at: www.microfocus.com Like what you read? Share it. Bundle Name Protocols Remote Access Citrix ICA
    [Show full text]
  • Continuous Quality and Testing to Accelerate Application Development
    Continuous Quality and Testing to Accelerate Application Development How to assess your current testing maturity level and practice continuous testing for DevOps Continuous Quality and Testing to Accelerate Application Development // 1 Table of Contents 03 Introduction 04 Why Is Continuous Quality and Testing Maturity Important to DevOps? 05 Continuous Testing Engineers Quality into DevOps 07 Best Practices for Well- Engineered Continuous Testing 08 Continuous Testing Maturity Levels Level 1: Chaos Level 2: Continuous Integration Level 3: Continuous Flow Level 4: Continuous Feedback Level 5: Continuous Improvement 12 Continuous Testing Maturity Assessment 13 How to Get Started with DevOps Testing? 14 Continuous Testing in the Cloud Choosing the right tools for Continuous Testing On-demand Development and Testing Environments with Infrastructure as Code The Right Tests at the Right Time 20 Get Started 20 Conclusion 21 About AWS Marketplace and DevOps Institute 21 Contributors Introduction A successful DevOps implementation reduces the bottlenecks related to testing. These bottlenecks include finding and setting up test environments, test configurations, and test results implementation. These issues are not industry specific. They can be experienced in manufacturing, service businesses, and governments alike. They can be reduced by having a thorough understanding and a disciplined, mature implementation of Continuous Testing and related recommended engineering practices. The best place to start addressing these challenges is having a good understanding of what Continuous Testing is. Marc Hornbeek, the author of Engineering DevOps, describes it as: “A quality assessment strategy in which most tests are automated and integrated as a core and essential part of DevOps. Continuous Testing is much more than simply ‘automating tests.’” In this whitepaper, we’ll address the best practices you can adopt for implementing Continuous Quality and Testing on the AWS Cloud environment in the context of the DevOps model.
    [Show full text]
  • Roman Udka Software Test Automation Engineer
    Roman Udka Software Test Automation Engineer Location: Ukraine, Kharkiv Telegram/Cell phone: +380994365885 E-Mail: [email protected] Summary: ● Experience 7 years. ● Performing execution and analysis test results. ● Implementation of qa education processes. ● Developing from scratch an automation framework based on the java stack. ● Organizing automation qa processes. ● Setuping CI/CD processes. ● Preparation approach for performance testing. ● Team leading / management at least 5 people; Skills: ● Test automation: Selenium (Selenide), TestNG, Selenium Grid (Selenoid). ● Performance tools: Locust, Gatling. ● Build automation/Continuous integration: Docker, Gradle, GitLabCi, Jenkins. ● Cloud: Azure, Linode. ● Bug Tracking/Test Management: Atlassian JIRA. ● Web debugging tools: Chrome dev tools. ● Programming: Java, Python; ● Version control system: Git; ● Agile: Scrum, Kanban. Work Experience: January, 2019 - till now Senior Software Test Engineer, GlobalLogic Project Role: Lead Automation Tester Tasks and Accomplishments: ● Automation tests creation and execution on 5 sub-projects. ● Performance testing. ● Creation of test automation framework from scratch and maintenance. ● Creation a test plan and strategy. ● Leading and supporting work of qa team; Environment: ● Selenide, TestNG, Git, Gradle, GitLabCI, IntelliJ IDEA, Azure, Report Portal. ● Java, Python, Selenium Grid, Docker, JIRA, DataDog, SendGrid, Locust, Gatling; January, 2015 - January, 2019 Software Test Engineer, EPAM Systems Project Role: Automation Tester Tasks and Accomplishments:
    [Show full text]
  • Loadrunner Cloud an Essential Component of the Devops Pipeline
    Product Flyer Application Delivery Management LoadRunner Cloud an Essential Component of the DevOps Pipeline Enable faster release and deployment cycles by taking advantage of agile development methodologies to achieve automated con- tinuous delivery. Micro Focus® LoadRunner Cloud makes it simple to integrate testing into the Development Process and test early and often. LoadRunner Cloud at a Glance: Applying an effective testing process within a Scalability in a Continuous high-speed delivery cycle can be a major chal- ■ Scalability: Testing Environment lenge, and traditional non-automated testing Managing hardware for performance testing A cloud-based solution that eliminates the methods may become a bottleneck. Con tin- tools is always a challenge. It becomes an even dependency on hardware as infrastructure is uous testing helps overcome this problem by bigger challenge in advanced DevOps environ- provisioned in the cloud fully automating testing throughout the delivery ments that have multiple teams and multiple ■ cycle. Micro Focus LoadRunner Cloud is the Automation: pipelines executing performance tests—with ideal solution to help overcome the potential Built in integration with Git, so scripts are managed challenges of applying continuous load testing. the need to manage a larger pool of load gener- in the repository for automatic updates prior to runs ators and controllers. LoadRunner Cloud, elim- ■ inates the dependency on hardware, allowing Collaboration: LoadRunner Cloud is the market-leading, multiple users and teams to connect to a single Project Management support, allowing users to share cloud-based performance testing solution, assets such as scripts, licenses, and load generators designed to support agile teams and DevOps cloud solution.
    [Show full text]
  • Integrating Performance Testing in Your CI/CD Process
    Product Flyer Application Delivery Management Integrating Performance Testing in your CI/CD Process Micro Focus® LoadRunner Cloud Makes It Insanely Simple to Integrate Testing into the Development Process—So You Can Test Early and Often. Everything You Need to Make Solution Overview start the testing process earlier, it’s still difficult Automated Testing Easy There’s no question: Continuous performance to actually build performance testing into the testing is critical to the success of your mobile deployment pipeline. ■ Simple, smart, scalable: and web apps. If you don’t test until the end of SaaS-based, pay-per-virtual-user model allows you the cycle—or if you don’t test at all—you risk Performance testing also needs to add value to get started with just a browser and a credit card. production issues, user complaints, poor re- to the business, not just development teams. The software provides graphical, realistic scripting views, and damage to your brand. It needs to improve the quality of the app, cus- and intuitive dashboards. It easily scales up to one tomer satisfaction levels, and the brand image. million virtual users without requiring hardware. The question is how to make continuous per- For application teams, that means the mental- ■ Enterprise-ready: formance and load testing practical in an age ity must change from simple “record/playback” With cross-team project management capabilities, of ever-accelerating development cycles and testing that occurs late in the product cycle to a it supports open APIs for easier enterprise-wide ever-increasing user expectations. Even if more robust engineering approach that starts integration.
    [Show full text]
  • Continuous Load Testing: Reinventing Load Testing for Devops
    Continuous Load Testing: Reinventing Load Testing for DevOps tricentis.com Executive Overview Transformational initiatives such as Agile and DevOps are compelling software development teams to change their approach to load testing. This new approach involves extending beyond late-cycle load testing by performance testing specialists and empowering developers and testers to perform early, continuous load testing. Flood IO is built to enable load testing for DevOps, helping DevTest teams gain performance feedback rapidly and continuously. In addition to supporting industry-leading load testing tools, Flood integrates with Tricentis Tosca—allowing users to perform load testing with the same scriptless tests they use for functional testing. For DevOps, Load Testing Shifts Left Over 91% of organizations already have Agile or DevOps initiatives or plan to adopt them within the immediate future. As part of the process transformation required to adopt these more iterative and rapid methods of delivering software, organizations need to reassess ALL practices associated with software development and test—including load and performance testing. Traditionally, load testing has been deferred until the late phases of waterfall release cycles. Specialized performance testers applied legacy load testing tools to validate application performance prior to release. Why is this no longer sufficient? • With teams moving to more componentized applications (including cloud-native apps, microservices, etc.), an application involves many highly-distributed components—and
    [Show full text]
  • Modern Performance Tools Applied
    Masaryk University Faculty of Informatics A Modern Performance Tools Applied Diploma thesis Ron Šmeral Brno, autumn 2014 Declaration Hereby I declare, that this paper is my original authorial work, which I have worked out by my own. All sources, references and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Ron Šmeral Advisor: Mgr. Marek Grác, Ph.D. ii Acknowledgement I would like to thank the supervisor of my diploma thesis, Mgr. Marek Grác, Ph.D., for giving me valuable advice and the confidence to finish the thesis in a restricted time frame. My thanks also go to my family and friends for their great patience and support. iii Abstract The goal of this thesis is to explore the state of the art in performance testing of web applications and to compare tools designed for that pur- pose. Five tools are investigated: Faban, Gatling, Grinder, JMeter and PerfCake. They are compared subjectively in several categories: architec- ture, protocol support, workload definition capabilities, injection profile support, usability, monitoring and reporting capabilities. The chosen tools were subjected to analysis of their performance characteristics in four test scenarios: HTTP GET request, REST GET request, JMS request-response and SOAP invocation. Two of the tools – Gatling and PerfCake – were chosen for further analysis in a test of a real application. iv Keywords performance testing, performance evaluation, stress testing, load testing, workload modelling v Contents 1 Introduction ...........................1 2 Performance testing ......................3 2.1 Concepts ...........................5 2.1.1 Transaction . .5 2.1.2 Workload modelling .
    [Show full text]
  • Gatling Apache Jmeter () Locust.Io ( Artillery.Io ()
    Software Architecture Software Architecture of Oviedo of Lab. 11 Load testing University , Other tests… Science Computer of of 2018-19 Jose Emilio Labra Gayo + Víctor Álvarez School [email protected] Software Architecture What are load tests? Measure performance under normal or anticipated peak load conditions Example: Several concurrent users Goal: Anticipate possible failures of Oviedo of verify work load of some system University , Science Computer of of School Software Architecture What can we test Web applications (Http/https) SOAP/REST Web Services FTP Databases (JDBC) of Oviedo of LDAP University , Mail (SMTP, POP3, IMAP) Science Java Objects Computer Etc. of of School Software Architecture Why should we do load tests? Anticipate performance problems Detect bottlenecks Prove quality attributes of Oviedo of University , Science Computer of of School Software Architecture Load testing tools Gatling Apache Jmeter () Locust.io (http://locust.io/) Artillery.io () of Oviedo of goReplay Loader.io University , BlazeMeter Science Blitz … Computer of of Step by step guide: School https://github.com/pglez82/docker_solid_example/tree/pglez82-gattling-load-tests#load-tests-gatling Software Architecture Gatling Written in Scala JVM compatible Embedded DSL for testing Easy to use of Oviedo of Light University , Science Computer of of School Software Architecture Download & installation http://gatling.io It needs Java 8 installed 2 scripts: Recorder.sh/Recorder.bat of Oviedo of Gatling.sh/Gatling.bat University , Science Computer of of School
    [Show full text]
  • Technology Profile
    2021 Technology Profile https://azati.ai +375 (29) 6845855 Belarus, 31 K. Marks Street, Sections 5-6 Grodno, 230025 1 Table Of Contents TABLE OF CONTENTS page 01 DEPLOYMENT, BI & DATA page 09 WAREHOUSING GENERAL INFORMATION page 02 DATA SCIENCE & MACHINE LEARNING page 10 JAVA TECHNOLOGIES page 03 MONITORING TOOLS, PORTALS & SOLUTIONS, page 11 VERSION CONTROL RUBY & JAVASCRIPT TECHNOLOGIES page 04 VERSION CONTROL, SDK & OTHER TOOLS page 12 WEB & PHP TECHNOLOGIES page 05 OTHER TOOLS page 13 MOBILE DEVELOPMENT & DATABASES page 06 SOFTWARE TESTING & QA page 07 APPLICATION DEPLOYMENT page 08 2 General Information 01 PROGRAMMING LANGUAGES: 02 MARK-UP AND MODELING 05 SOFTWARE ARCHITECTURE PATTERNS: LANGUAGES: Java Representational State Transfer (REST/RESTful) JavaScript (ES5/ES6) HTML (4/5) Model-View-Controller (MVC) PHP XSLT Microservices TypeScript UML GraphQL PL/SQL Kotlin Smalltalk C 03 PROJECT MANAGEMENT C++ METHODOLOGIES: C# Agile (Kanban/SCRUM) Groovy Waterfall Delphi Behavior-driven development (BDD) Pascal Test-driven development (TDD) Python Feature-driven development (FDD) SQL Ruby R CoffeeScript 04 DEVELOPMENT APPROACHES: Perl Continuous Delivery (CD) Bash Continuous Integration (CI) Shell 3 Java Technologies 06 JAVA TECHNOLOGIES: 07 JAVA FRAMEWORKS: Apache POI Java (7/8/9) Spring Apache Wicket Java Servlet Spring Boot Apache CXF Java Database Connectivity (JDBC) Spring REST Apache Shiro Java REST Spring MVC Apache Camel Java Persistence API (JPA) Spring Data Java Message Service (JMS) Spring Security 08 JAVA LIBRARIES: JBoss Drools
    [Show full text]
  • A Reactive Platform for Real-Time Asset Trading Case Study | Softwaremill
    A reactive platform for real-time asset trading Case Study CLIENT PROFILE Our client is a marketplace for buying and selling many currencies as well as ICO (Initial Coin Offering). PROJECT OVERVIEW INDUSTRY FINTECH GOAL OF THE PROJECT IMPLEMENTING CORE ENGINE FOR A DIGITAL ASSET EXCHANGE SYSTEM CHALLENGE HANDLE UP TO 10000 REQUESTS PER SECOND IN A TRADING PLATFORM PROJECT DURATION 2.5 YEARS, INCLUDING 10 MONTHS TO REWRITE THE TRADING SERVICE TECHNOLOGIES SCALA, AKKA PERSISTENCE, AKKA CLUSTER, AKKA STREAMS, CASSANDRA (DSE), APACHE KAFKA, GATLING, PROMETHEUS, GRAYLOG, DC/OS BENEFITS REACTIVE AND RELIABLE EXCHANGE SYSTEM WITH INCREASING VOLUME EVENT STORE KEEPING ALL BUSINESS EVENTS, ENABLING REAL-TIME INSIGHTS AND REPORTING MODULARITY ALLOWING KEEPING THE EXCHANGE BUSINESS RUNNING WHILE UPDATING OTHER COMPONENTS BACKGROUND Our customers' system was a very successful one, mainly because it has been developed in a very short period of time, providing key features like limit offers, market offers, extended history search, and other important capabilities. Exposed as an elegant and robust web interface as well in a form of accessible REST API, the platform quickly became the main source of transactions for traders all over the world. As the user base started to expand, the company has been frequently adding new attractive markets. This, combined with solid marketing oriented on trustworthiness and reliability, has driven the company to an extraordinary success. However, it has become more and more expensive for the company to handle failures manually. Additionally, a very high growth of users and markets started to show that performance capabilities are limited. Because of very simple design, the system was very well doing its job, but only to a point when it became necessary to scale it.
    [Show full text]