Perfguard: Binary-Centric Application Performance Monitoring in Production Environments

Total Page:16

File Type:pdf, Size:1020Kb

Perfguard: Binary-Centric Application Performance Monitoring in Production Environments PerfGuard: Binary-Centric Application Performance Monitoring in Production Environments Chung Hwan Kimy Junghwan Rheez Kyu Hyung Lee{ Xiangyu Zhangy Dongyan Xuy yPurdue University zNEC Laboratories America {University of Georgia West Lafayette, IN, USA Princeton, NJ, USA Athens, GA, USA {chungkim, xyzhang, [email protected] [email protected] dxu}@cs.purdue.edu ABSTRACT other software issues, preventing performance problems be- Diagnosis of performance problems is an essential part of fore software distribution is challenging [42] for the following software development and maintenance. This is in partic- reasons. First, modern software has complex dependency ular a challenging problem to be solved in the production on many components developed by multiple parties. For ex- environment where only program binaries are available with ample, an application may have dependency on third-party limited or zero knowledge of the source code. This problem libraries as well as the system libraries to use the underlying is compounded by the integration with a significant number operating system. Therefore, finding the root causes of per- of third-party software in most large-scale applications. Ex- formance problems requires investigation of the whole soft- isting approaches either require source code to embed man- ware stack of various software component layers [43]. Sec- ually constructed logic to identify performance problems or ond, it is very challenging to identify performance issues support a limited scope of applications with prior manual during the development because software vendors have lim- analysis. This paper proposes an automated approach to ited time and environments to test various complex usage analyze application binaries and instrument the binary code scenarios. Consequently, there have been efforts to diagnose transparently to inject and apply performance assertions on performance problems during production deployment, long application transactions. Our evaluation with a set of large- after the development stage [43, 38, 32, 54]. scale application binaries without access to source code dis- Production-run performance diagnosis has been performed covered 10 publicly known real world performance bugs au- generally in two major ways, which complement each other tomatically and shows that PerfGuard introduces very low and often are used together. First, software vendors main- overhead (less than 3% on Apache and MySQL server) to tain bug reporting systems [15, 12, 23]. These systems are production systems. used for reporting software issues such as performance and failures issues. Users can voluntarily report the details of their performance issues, for instance, how to reproduce the CCS Concepts symptom, the specifications of their system, etc. Second, •Software and its engineering ! Software perfor- some software vendors embed code logic to detect unex- mance; Software testing and debugging; Software pected performance delay and to report the incident to the post-development issues; vendors automatically [16]. Specifically such logic monitors the performance of semantically individual operations of a 1 Keywords program and raises an alarm if their latency exceeds pre- determined thresholds. However, the cost of human efforts Performance diagnosis, post-development testing to support such logic and thresholds is high due to require- ments to perform in-depth analysis on possible application 1. INTRODUCTION behaviors and to determine the range of its reasonable execu- Diagnosis and troubleshooting of performance problems is tion time. In addition, the location to insert the logic needs an essential part of software development and maintenance. to be manually determined considering its functionality and Traditionally, various performance tools [22, 14, 36, 21, 13, run-time impact. Such manual efforts may involve human 34] have been extensively used by developers during the de- errors due to the misunderstanding of complex program be- velopment and testing stages in order to identify inefficient haviors, particularly when dealing with large-scale software. code and prevent performance problems. However, unlike Although automating the process could save significant ef- forts in performance debugging and testing, such feature is not implemented by many software vendors in practice. Permission to make digital or hard copies of all or part of this work for personal or Furthermore, software users at the deployment stage re- classroom use is granted without fee provided that copies are not made or distributed quire performance diagnostics for production software with- for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM out source code or deep knowledge of the target application. must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, For instance, service providers use open source programs to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. 1 FSE’16, November 13–18, 2016, Seattle, WA, USA Such operations are also known as application transactions, c 2016 ACM. 978-1-4503-4218-6/16/11...$15.00 user transactions [54], units [44], or business transactions [5, http://dx.doi.org/10.1145/2950290.2950347 19, 10]. We will use application transactions herein. 595 or 3rd-party software as part of their large infrastructure. Unit and Metadata Instrumenting App Performance Guard Application with Monitoring their performance in the production stage is im- Identification Performance Guards portant due to their impact on the overall quality of the Pre-distribution service. However, the lack of source code and code knowl- Feedback Production-run Deploy Unit Performance Unit Performance edge for instrumentation make the monitoring challenging. Monitoring Inspection Trigger There are application performance management (APM) ser- vices available in the market [5, 19, 10], but those services Figure 1: PerfGuard Architecture. require developers to modify the application code for insert- ing the monitoring API provided by the service or support a • Implementation and evaluation of a prototype with limited range of known applications and platforms with prior various applications in the Microsoft Windows plat- analysis. Table 1 lists the APM services and existing tech- form showing its practicality discovering performance niques for application performance diagnosis in comparison. bugs automatically in the real world software. Among the compared approaches, only PerfGuard automat- ically identifies application transactions and monitors their x2 presents the design overview of PerfGuard. The key performance with no code knowledge using only binaries. idea of unit-based performance analysis is presented in x3. x4 To provide a generally applicable, binary-centric frame- shows how to identify units and performance guards. Instru- work for performance monitoring, we designed and imple- mentation of applications with performance guards is pre- mented PerfGuard, which automatically transforms the bi- sented in x5. Implementation and evaluation of PerfGuard naries of an application without its source code or prior code are presented in x6 and x7. x8 and x9 respectively show knowledge to monitor its performance at the production run. related work and discussions. x10 concludes this paper. PerfGuard performs dynamic analysis of a target applica- tion and generates a performance profile, which is a \sum- 2. DESIGN OVERVIEW mary" of the application performance behavior. It is used The overall architecture of PerfGuard is presented in Fig- as a \hint" to create a set of monitoring code in the binary ure 1. PerfGuard takes the binaries of a target application format, called performance guards, to be inserted into the as an input and automatically discovers application trans- binaries of the application to diagnose its performance in actions from a set of training runs to produce a perfor- the production environment. During the production run, mance profile incrementally. The preciseness of the perfor- the inserted performance guards automatically monitor the mance profile increases as more training runs are performed. performance of specific application transactions that are au- By analyzing the performance profile PerfGuard generates tomatically determined by our analysis. An unexpected per- a set of binary code snippets, called performance guards, formance delay triggers a performance assertion set by the that are injected into the binaries on the discovered applica- performance guard, and invokes a performance diagnosis to tion transactions to monitor application performance. While help developers and users resolve the issue. the instrumented application is in the production stage, the PerfGuard utilizes program execution partitioning (a.k.a. performance guards detect potential performance anomalies units [44]) to automate the recognition of application trans- and inspect the application's state to find their root causes. actions. Each unit is a segment of the execution of a process To discover application transactions, PerfGuard leverages that handles a specific workload. During the profiling of the the fact that a majority of large-scale applications are event- application, the units of the identical or similar control flow driven and incorporate a small number of loops in each event are clustered into the same unit type. PerfGuard determines
Recommended publications
  • Effectiveness of Software Testing Techniques in Enterprise: a Case Study
    MYKOLAS ROMERIS UNIVERSITY BUSINESS AND MEDIA SCHOOL BRIGITA JAZUKEVIČIŪTĖ (Business Informatics) EFFECTIVENESS OF SOFTWARE TESTING TECHNIQUES IN ENTERPRISE: A CASE STUDY Master Thesis Supervisor – Assoc. Prof. Andrej Vlasenko Vilnius, 2016 CONTENTS INTRODUCTION .................................................................................................................................. 7 1. THE RELATIONSHIP BETWEEN SOFTWARE TESTING AND SOFTWARE QUALITY ASSURANCE ........................................................................................................................................ 11 1.1. Introduction to Software Quality Assurance ......................................................................... 11 1.2. The overview of Software testing fundamentals: Concepts, History, Main principles ......... 20 2. AN OVERVIEW OF SOFTWARE TESTING TECHNIQUES AND THEIR USE IN ENTERPRISES ...................................................................................................................................... 26 2.1. Testing techniques as code analysis ....................................................................................... 26 2.1.1. Static testing ...................................................................................................................... 26 2.1.2. Dynamic testing ................................................................................................................. 28 2.2. Test design based Techniques ...............................................................................................
    [Show full text]
  • Studying the Feasibility and Importance of Software Testing: an Analysis
    Dr. S.S.Riaz Ahamed / Internatinal Journal of Engineering Science and Technology Vol.1(3), 2009, 119-128 STUDYING THE FEASIBILITY AND IMPORTANCE OF SOFTWARE TESTING: AN ANALYSIS Dr.S.S.Riaz Ahamed Principal, Sathak Institute of Technology, Ramanathapuram,India. Email:[email protected], [email protected] ABSTRACT Software testing is a critical element of software quality assurance and represents the ultimate review of specification, design and coding. Software testing is the process of testing the functionality and correctness of software by running it. Software testing is usually performed for one of two reasons: defect detection, and reliability estimation. The problem of applying software testing to defect detection is that software can only suggest the presence of flaws, not their absence (unless the testing is exhaustive). The problem of applying software testing to reliability estimation is that the input distribution used for selecting test cases may be flawed. The key to software testing is trying to find the modes of failure - something that requires exhaustively testing the code on all possible inputs. Software Testing, depending on the testing method employed, can be implemented at any time in the development process. Keywords: verification and validation (V & V) 1 INTRODUCTION Testing is a set of activities that could be planned ahead and conducted systematically. The main objective of testing is to find an error by executing a program. The objective of testing is to check whether the designed software meets the customer specification. The Testing should fulfill the following criteria: ¾ Test should begin at the module level and work “outward” toward the integration of the entire computer based system.
    [Show full text]
  • Visual Studio Team Test Quick Reference a Quick Reference for Users of the Team Testing Features of Visual Studio Team System
    MICROSOFT Visual Studio Team Test Quick Reference A quick reference for users of the Team Testing features of Visual Studio Team System Geoff Gray and the Microsoft VSTS Rangers team 3/30/2009 VSTS Rangers This content was originally created by Geoff Gray for internal Microsoft use and then adopted and expanded as a Visual Studio Team System (“VSTS”) Rangers project. “Our mission is to accelerate the adoption of Team System by delivering out of band solutions for missing features or guidance. We work closely with members of Microsoft Services to make sure that our solutions address real world blockers.” -- Bijan Javidi, VSTS Rangers Lead Copyright 2009 Microsoft Corporation Page | 1 Summary This document is a collection of items from public blog sites, Microsoft® internal discussion aliases (sanitized) and experiences from various Test Consultants in the Microsoft Services Labs. The idea is to provide quick reference points around various aspects of Microsoft Visual Studio® Team Test edition that may not be covered in core documentation, or may not be easily understood. The different types of information cover: How does this feature work under the covers? How can I implement a workaround for this missing feature? This is a known bug and here is a fix or workaround. How do I troubleshoot issues I am having? The document contains two Tables of Contents (high level overview, and list of every topic covered) as well as an index. The current plan is to update the document on a regular basis as new information is found. The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication.
    [Show full text]
  • Manual on Quality Assurance for Computer Software Related to the Safety of Nuclear Power Plants
    SIMPLIFIED SOFTWARE LIFE-CYCLE DIAGRAM FEASIBILITY STUDY PROJECT TIME I SOFTWARE P FUNCTIONAL I SPECIFICATION! SOFTWARE SYSTEM DESIGN DETAILED MODULES CECIFICATION MODULES DESIGN SOFTWARE INTEGRATION AND TESTING SYSTEM TESTING ••COMMISSIONING I AND HANDOVER | DECOMMISSION DESIGN DESIGN SPECIFICATION VERIFICATION OPERATION AND MAINTENANCE SOFTWARE LIFE-CYCLE PHASES TECHNICAL REPORTS SERIES No. 282 Manual on Quality Assurance for Computer Software Related to the Safety of Nuclear Power Plants f INTERNATIONAL ATOMIC ENERGY AGENCY, VIENNA, 1988 MANUAL ON QUALITY ASSURANCE FOR COMPUTER SOFTWARE RELATED TO THE SAFETY OF NUCLEAR POWER PLANTS The following States are Members of the International Atomic Energy Agency: AFGHANISTAN GUATEMALA PARAGUAY ALBANIA HAITI PERU ALGERIA HOLY SEE PHILIPPINES ARGENTINA HUNGARY POLAND AUSTRALIA ICELAND PORTUGAL AUSTRIA INDIA QATAR BANGLADESH INDONESIA ROMANIA BELGIUM IRAN, ISLAMIC REPUBLIC OF SAUDI ARABIA BOLIVIA IRAQ SENEGAL BRAZIL IRELAND SIERRA LEONE BULGARIA ISRAEL SINGAPORE BURMA ITALY SOUTH AFRICA BYELORUSSIAN SOVIET JAMAICA SPAIN SOCIALIST REPUBLIC JAPAN SRI LANKA CAMEROON JORDAN SUDAN CANADA KENYA SWEDEN CHILE KOREA, REPUBLIC OF SWITZERLAND CHINA KUWAIT SYRIAN ARAB REPUBLIC COLOMBIA LEBANON THAILAND COSTA RICA LIBERIA TUNISIA COTE D'lVOIRE LIBYAN ARAB JAMAHIRIYA TURKEY CUBA LIECHTENSTEIN UGANDA CYPRUS LUXEMBOURG UKRAINIAN SOVIET SOCIALIST CZECHOSLOVAKIA MADAGASCAR REPUBLIC DEMOCRATIC KAMPUCHEA MALAYSIA UNION OF SOVIET SOCIALIST DEMOCRATIC PEOPLE'S MALI REPUBLICS REPUBLIC OF KOREA MAURITIUS UNITED ARAB
    [Show full text]
  • Download Full CV (PDF)
    Full name : Lars Bjergner Mikkelsen. Practical experience: Company name: LARSMIKKELSEN.COM Aps Street & number: Husoddebakken 26 City: Horsens Zip code: 8700 Country: Denmark. Web address: http://www.larsmikkelsen.com Start date of employment: 27-July-2007. End date of employment: Not ended Job title: Owner at LARSMIKKELSEN.COM Aps. Job description: Freelance specialist Microsoft Dynamics Ax and .NET. Technical solution architect Dynamics Ax projects. Development in x++ and C#. Integration specialist between Dynamics Ax and .NET on several projects. SharePoint Enterprise Portal solutions on Dynmaics Ax 4.0 and ASP.NET based Dynamics Ax 2009 solution. Invented, designed and developed Advanced Ax Batch. Advanced Ax Batch is a Dynamics Ax and .NET based scheduler which are used by several companies for batch execution in Dynamcis Ax. Performance optimization Dynamics Ax solutions. Specialized knowledge: Highly experienced with performance optimization and trouble shooting of Dynamics Ax installations. Technologies mastered: Programming Languages (X++, C#) Programming Libraries (Axapta, .NET Framework) Component Technology (Axapta, .NET , COM, COM+, Active X) Databases (SQL server) Markup Languages (HTML, XML) Internet (SharePoint Enterprise Portal) Development tools (Axapta, Visual studio .NET) Protocols (HTTP, SOAP, TCP/IP) 1 Company name: Columbus IT Street & number: 3151 Airway, Building N-1 City: Costa Mesa, CA Zip code: 8240 Country: USA. Web address: http://www.columbusit.com Start date of employment: 23-May-2005. End date of employment: 27-July-2007. Job title: Technology / integration manager and solution architect. Job description: Responsible for technology and integration strategies. Technical solution architect on major Dynamics Ax projects. Development in x++ and C#. Technical responsible for worldwide mobility platform.
    [Show full text]
  • Test-Driven Development in Enterprise Integration Projects
    Test-Driven Development in Enterprise Integration Projects November 2002 Gregor Hohpe Wendy Istvanick Copyright ThoughtWorks, Inc. 2002 Table of Contents Summary............................................................................................................. 1 Testing Complex Business Applications......................................................... 2 Testing – The Stepchild of the Software Development Lifecycle?............................................... 2 Test-Driven Development............................................................................................................. 2 Effective Testing........................................................................................................................... 3 Testing Frameworks..................................................................................................................... 3 Layered Testing Approach ........................................................................................................... 4 Testing Integration Solutions............................................................................ 5 Anatomy of an Enterprise Integration Solution............................................................................. 5 EAI Testing Challenges................................................................................................................ 6 Functional Testing for Integration Solutions................................................................................. 7 EAI Testing Framework ..................................................................................
    [Show full text]
  • Continuous Quality and Testing to Accelerate Application Development
    Continuous Quality and Testing to Accelerate Application Development How to assess your current testing maturity level and practice continuous testing for DevOps Continuous Quality and Testing to Accelerate Application Development // 1 Table of Contents 03 Introduction 04 Why Is Continuous Quality and Testing Maturity Important to DevOps? 05 Continuous Testing Engineers Quality into DevOps 07 Best Practices for Well- Engineered Continuous Testing 08 Continuous Testing Maturity Levels Level 1: Chaos Level 2: Continuous Integration Level 3: Continuous Flow Level 4: Continuous Feedback Level 5: Continuous Improvement 12 Continuous Testing Maturity Assessment 13 How to Get Started with DevOps Testing? 14 Continuous Testing in the Cloud Choosing the right tools for Continuous Testing On-demand Development and Testing Environments with Infrastructure as Code The Right Tests at the Right Time 20 Get Started 20 Conclusion 21 About AWS Marketplace and DevOps Institute 21 Contributors Introduction A successful DevOps implementation reduces the bottlenecks related to testing. These bottlenecks include finding and setting up test environments, test configurations, and test results implementation. These issues are not industry specific. They can be experienced in manufacturing, service businesses, and governments alike. They can be reduced by having a thorough understanding and a disciplined, mature implementation of Continuous Testing and related recommended engineering practices. The best place to start addressing these challenges is having a good understanding of what Continuous Testing is. Marc Hornbeek, the author of Engineering DevOps, describes it as: “A quality assessment strategy in which most tests are automated and integrated as a core and essential part of DevOps. Continuous Testing is much more than simply ‘automating tests.’” In this whitepaper, we’ll address the best practices you can adopt for implementing Continuous Quality and Testing on the AWS Cloud environment in the context of the DevOps model.
    [Show full text]
  • Microsoft SQL Server Analysis Services Multidimensional Performance and Operations Guide Thomas Kejser and Denny Lee
    Microsoft SQL Server Analysis Services Multidimensional Performance and Operations Guide Thomas Kejser and Denny Lee Contributors and Technical Reviewers: Peter Adshead (UBS), T.K. Anand, KaganArca, Andrew Calvett (UBS), Brad Daniels, John Desch, Marius Dumitru, WillfriedFärber (Trivadis), Alberto Ferrari (SQLBI), Marcel Franke (pmOne), Greg Galloway (Artis Consulting), Darren Gosbell (James & Monroe), DaeSeong Han, Siva Harinath, Thomas Ivarsson (Sigma AB), Alejandro Leguizamo (SolidQ), Alexei Khalyako, Edward Melomed, AkshaiMirchandani, Sanjay Nayyar (IM Group), TomislavPiasevoli, Carl Rabeler (SolidQ), Marco Russo (SQLBI), Ashvini Sharma, Didier Simon, John Sirmon, Richard Tkachuk, Andrea Uggetti, Elizabeth Vitt, Mike Vovchik, Christopher Webb (Crossjoin Consulting), SedatYogurtcuoglu, Anne Zorner Summary: Download this book to learn about Analysis Services Multidimensional performance tuning from an operational and development perspective. This book consolidates the previously published SQL Server 2008 R2 Analysis Services Operations Guide and SQL Server 2008 R2 Analysis Services Performance Guide into a single publication that you can view on portable devices. Category: Guide Applies to: SQL Server 2005, SQL Server 2008, SQL Server 2008 R2, SQL Server 2012 Source: White paper (link to source content, link to source content) E-book publication date: May 2012 200 pages This page intentionally left blank Copyright © 2012 by Microsoft Corporation All rights reserved. No part of the contents of this book may be reproduced or transmitted in any form or by any means without the written permission of the publisher. Microsoft and the trademarks listed at http://www.microsoft.com/about/legal/en/us/IntellectualProperty/Trademarks/EN-US.aspx are trademarks of the Microsoft group of companies. All other marks are property of their respective owners.
    [Show full text]
  • Devops Point of View an Enterprise Architecture Perspective
    DevOps Point of View An Enterprise Architecture perspective Amsterdam, 2020 Management summary “It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change.”1 Setting the scene Goal of this Point of View In the current world of IT and the development of This point of view aims to create awareness around the IT-related products or services, companies from transformation towards the DevOps way of working, to enterprise level to smaller sizes are starting to help gain understanding what DevOps is, why you need it use the DevOps processes and methods as a part and what is needed to implement DevOps. of their day-to-day organization process. The goal is to reduce the time involved in all the An Enterprise Architecture perspective software development phases, to achieve greater Even though it is DevOps from an Enterprise Architecture application stability and faster development service line perspective, this material has been gathered cycles. from our experiences with customers, combined with However not only on the technical side of the knowledge from subject matter experts and theory from organization is DevOps changing the playing within and outside Deloitte. field, also an organizational change that involves merging development and operations teams is Targeted audience required with an hint of cultural changes. And last but not least the skillset of all people It is specifically for the people within Deloitte that want to involved is changing. use this as an accelerator for conversations and proposals & to get in contact with the people who have performed these type of projects.
    [Show full text]
  • Kindle \\ Windows Software Introduction « Download
    Windows software Introduction < PDF / 6UJIY3QLQT Windows software Introduction By Source Reference Series Books LLC Jun 2011, 2011. Taschenbuch. Book Condition: Neu. 246x192x10 mm. Neuware - Source: Wikipedia. Pages: 64. Chapters: MediaMonkey, Microsoft SharePoint Designer, Microsoft Office 95, Genbox Family History, EMC Legato NetWorker, IObit Security 360, DoPDF, Conserver, RootsMagic, Intel Parallel Studio, Exposé clone, Microsoft Office Accounting, IsoBuster, Microsoft Digital Image, DOSKey, Creo Elements/Direct, GoldWave, Universal Converter, Indigo Renderer, Jing, Internet Explorer 1, DirectVobSub, Adobe Flash Builder, MSDN Academic Alliance, Unidrv, Microsoft Write, Cadstar, DVD Flick, Winpooch, GoBinder, ICore Virtual Accounts, Contig, VSTS Profiler, Xplorer , Portrait Professional, Emergent, Bootsplash, STDU Explorer, UltraISO, Recuva, GCompris, WinEdt, ATOK, The Hessling Editor, SpeedCommander, FlexWiki, Brother's Keeper, Microsoft Live Labs Deepfish, Orbit Downloader, FreeCAD, BullsHit Converter, XYplorer, TUGZip, Take Command, AppLocale, Group Shot, Adobe Technical Communication Suite, DiskXtender, Network UPS Tools, SE-Explorer, SpywareBlaster, Adobe eLearning Suite, Foxy, TortoiseCVS, Adobe Version Cue, FlexPro, Altap Salamander, ConTEXT, Norton PC Checkup, Family Historian, Format, Dolphin Smalltalk, GenStat, Keynote, Resource Hacker, CDRWIN, Crystal Enterprise, RUMBA, Cartes du Ciel, Enigmail, Apcupsd, Biblioscape, ChemDraw, Pocomail, Glk, WinBUGS, Bandicam, EPrompter, Microsoft Vizact, SwordSearcher, HP NewWave, Microsoft
    [Show full text]
  • API Integration Tutorial: Testing, Security and API Management
    API Integration Tutorial: Testing, security and API management Tutorial In this tutorial In this tutorial: A roundup of the leading API As application program interface integration increases, so do the management tools available challenges with maintaining management, testing, and security. today………………………………..…. p.2 This API integration tutorial compares leading API management What are some solid options tools currently available on the market, as well as strategies for for open source API RESTful API testing. management tools?.............. p.14 Get a better understanding of API integration and management The basics of establishing a strategies. RESTful API testing program ……………………………………………. p.17 Testing microservices and APIs in the cloud……………... p.24 How do I create a secure API for mobile?............................... p.29 About SearchMicroservices.com ……………………………………………. p.31 Page 1 of 31 Tutorial In this tutorial A roundup of the leading API management A roundup of the leading API tools available today management tools available today………………………………..…. p.2 Zachary Flower, Freelance writer, SearchMicroservices.com API management is a constantly growing market with more new products What are some solid options popping up every year. As a result, narrowing all of the potential choices for open source API down to one perfect platform can feel like an overwhelming task. The sheer management tools?.............. p.14 number of options out there makes choosing one an extremely tough decision. The basics of establishing a To help relieve the strain involved with this decision, we've put together this RESTful API testing program detailed product roundup covering 10 of the leading API management tools ……………………………………………. p.17 currently available on the market.
    [Show full text]
  • Accessibility Testing in Agile Software Development
    Accessibility Testing in Agile Software Development Empowering agile developers with accessibility methods Nikolai Sverdrup Thesis submitted for the degree of Master in Programming and Networks 60 credits Department of Informatics Faculty of mathematics and natural sciences UNIVERSITY OF OSLO Autumn 2018 Accessibility Testing in Agile Software Development Empowering agile developers with accessibility methods Nikolai Sverdrup c 2018 Nikolai Sverdrup Accessibility Testing in Agile Software Development http://www.duo.uio.no/ Printed: Reprosentralen, University of Oslo Abstract This thesis delves into some of the methods that can be used to do accessibility testing in software development. Challenging traditional conventions of working for accessibility in agile development, where slow and resource intensive methods dominate, I investigate several methods that could ease the task of creating accessible software. I will mainly be focusing on web accessibility in tandem with a wider world effort working on the same goal. I have deployed several methods to professional software developers working in many different specialties. There are five methods I deploy and evaluate: A type of glasses that emulate bad eyesight, a software tool that analyzes and reports accessibility faults on websites, Screen-readers used a debugging tool, a method where the testers act out the user experience of imagined disabled users and a dyslectic emulation tool. There is also a smaller evaluation of using traditional WCAG testing. A case study was used to try and get a closer look at real life accessibility testing and aiding in finding out what obstacles accessibility testing need to climb if the testing methods are to be deployed successfully.
    [Show full text]