DOCSLIB.ORG

Ten Tips of Web App Testing How to Test and Launch a World-Class Application

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Ten Tips of Web App Testing How to Test and Launch a World-Class Application Ten Tips of Web App Testing How to test and launch a world-class application White Paper June 2011 White Paper: Web App Testing White Paper Web App Testing How to test and launch a world-class application Table of Contents • Introduction………………………………………….. 3 - New Challenges, Old Solutions…………..……… 3 - The Crowdsourcing Advantage………………….. 4 - Testing Types……………………………..……….. 4 • 10 Tips of Web App Testing………………………. 6 “Why is there never time 1. Don’t Be a Slave to Your Metrics………….. 6 2. Know Thy Third Party Apps……..…………. 7 to do it right, but always 3. The Testing Manager’s Dilemma………….. 7 time to do it over?” 4. Invest in Testers, Then Tools……………….. 8 5. Protect Users, Save Yourself……………….. 8 6. Multimedia, Multi-Problems….……………… 9 - Colonel Mike Mullane 7. Forfeit the Blame Game……………………… 9 Retired NASA Astronaut 8. Be Everywhere……………………………….. 10 9. Plugins: Proceed With Caution…………….. 10 10. Cross-Browser Testing……….……………… 10 • About uTest………………………………………….. 12 2 White Paper: Web App Testing Introduction New Challenges, Old Solutions for Testing Your Web Application Establishing procedure amidst a constant state of change may seem like an absurdity (and a losing battle), but this is what today’s software companies face when it comes to testing their web applications. Aside from an expanding matrix of new browsers, plugins, third party apps, programming languages and more, there are now thousands, if not millions, of new users from all around the globe. Of course, none of this was in the manual! “Complexity kills. It sucks the life out Thus, the purpose of this whitepaper is of developers, it makes products twofold. While our ‘10 Tips’ section will difficult to plan, build and test, it help you stay on top on of the latest introduces security challenges, and it trends in web app testing, our introduction will outline the basics of causes end-user and administrator functional, usability and load testing for frustration.” web-based applications. Along the - Ray Ozzie way, we’ll cover topics such as: Chief Software Architect, Microsoft Third Party Apps: Your product is only part of the equation - what about the satellite apps that are orbiting your web application? Cloud Computing: With opportunity comes risk. There’s a ton of upside to cloud computing, but what happens to your web app when it fails to functional properly? Multimedia: How will your web app perform for users with different versions of Flash, Acrobat, Java, etc.? Have you thought about HTML5 and the challenges of managing H.264 vs. Theora? Plugins: While the use of plugins may be slowing, the use of extensions are not. Learn why plugins and their extensions should be a testing focal point. Localization: Obviously, your application needs to work where your users reside. Discover why most software companies underestimate the importance of localization in their web testing. Deadlines: Your web application will never be perfect, so how will you know when it’s safe to launch? It’s called the “Testing Managers Dilemma” and we’ll show you how to deal with it. Metrics: As one testing expert put it, “When your car is about to go off a cliff, it’s a weird time to be thinking about gas mileage and drag coefficients.” Find out how metrics can mislead. 3 White Paper: Web App Testing The Crowdsourcing Advantage While the following material is suited for companies that rely on traditional staffing models for testing (i.e. internal QA teams or offshoring firms), this whitepaper will be especially useful for those who leverage some aspect of crowdsourced (or community- based) testing. To learn more about the growing trend of crowdsourcing - and how companies of all sizes can use it to complement their in-house efforts – check out the 8 Essentials of Crowdsourcing whitepaper. Web Applications: Testing Types “For a successful technology, Functional Testing reality must take precedence No surprises here. The most frequent and critical over public relations, for nature task that QA takes on before launch is to perform cannot be fooled comprehensive functional testing. This can .” include - but is not limited to - testing of: - Richard Feynman Renowned Scientist Web Forms: This includes sign-up, sign- in, contact and purchase forms; database queries, checking for cross-site scripting/AJAX vulnerabilities and other issues related to incorrect inputs. Links: Ensuring that all outgoing and internal links are functional and accurate. GUI: Despite back-end test automation, front end problems with web applications are almost always discovered by human testers, including issues related to browser compatibility or CSS flaws. Language: Does your web app properly support Unicode? Can it accept and handle foreign characters? This is a growing concern for developers and should be considered as an essential part of your testing efforts. Security: Security testing – including checks for open redirects, cookie tampering, SQL injections and other vulnerabilities that could harm your users (and reputation) – should also be performed prior to launch. Bottom line: Does your web application work when, where and how your users need it to work? Usability Testing Beta testing may be suitable for some large companies (mainly those with name recognition) who want to solicit user feedback, but it is NOT an effective substitute for usability testing, which should include professional analysis of: Graphics: Have you identified and addressed the major GUI bugs in your web app? Are the color schemes easy to understand (in every country of your user 4 White Paper: Web App Testing base)? Many of these flaws will not be reported by beta users, but they should be known about and fixed before launch. Feature Set: This can include the intuitiveness of shopping lists, online checkouts and other e-commerce actions; the accuracy of search results, data entry, sign-up forms and other features. Accessibility: Is your application accessible for all users, including the visually impaired? Does your app involve parental or administrator restrictions? Be aware of your legal obligations. Benchmarking: How does your web application stand up to that of your closest competitors? What about your own previous versions? What areas does it compare favorably or unfavorably? Overall Usability: Beta testers - if they report bugs at all - are likely to respond with “this-site-sucks” or “this-site-is-great” type of analysis. When launching a web application, professional, non-biased analysis is essential. Bottom line: Is your web application intuitive and highly usable for your target audience? Load Testing “The real value of tests is not How will your web application hold up when it that they detect bugs in the code, needs to most? Synthetic load tools are helpful in simulating these types of scenarios, but with the but that they detect inadequacies assistance of real testers, you can obtain a in the methods, concentration complete picture of how your application and skill of those who design and performs under stress. produce the code.” Load testing should, at the very least, involve one of the following methods: - Tony Hoare Testing Expert Live Load Testing: A team of live testers (preferably from where your users reside – whether it be North America or a global audience) will test your application simultaneously, enabling you to see how your app performs under real-world, non-simulated load. This process, as its name would indicate, requires NO automated tools. Ideal for web apps that contain Flash, streaming video or other types of multimedia that are difficult to evaluate with simulated load. Simulated Load Testing: Using the simulated load testing tool of your choice (more on this below) you can obtain a realistic snapshot of your web application's performance under peak synthetic usage. Hybrid Load Testing: A combination of the aforementioned methods, this process involves having live testers perform functional testing on your web application while under automated load. This lets you see bugs or performance issues that only show themselves under peak loads. 5 White Paper: Web App Testing Bottom line: Will your web application perform as expected under maximum pressure? Will it crash? Will the performance degrade? Web Testing Checklist: A Summary Regardless of whether you are performing functional, usability or load testing, it is imperative that each of these practices achieve maximum testing coverage that matches your user base across: Location: If you have a global user base, why would you only “When your car is about to go off a cliff, test your web application in your it’s a weird time to be thinking about office? It’s therefore critical to gas mileage and drag coefficients; extend your testing coverage better to take the right control action— beyond your borders if needed. look out the window and steer or use Language: If your application is available in multiple languages, the brake until you’re back on course.” you must verify that nothing is lost in translation (like the - Michael Bolton intuitiveness of your content, CEO, DevelopSense error messages and core features). Operating Systems: Your app needs to work seamlessly across all the different flavors and versions of Windows, Mac and Linux. Browsers: The days of Internet Explorer hegemony are over. Today’s web applications must be tested thoroughly against multiple versions of IE, Firefox, Chrome, Safari and others. And now, the Ten Tips of Web Testing…. The 10 Tips of Web Testing 1. Don’t Be a Slave to Your Metrics When testing a web application, it’s easy to become overwhelmed by the data that’s available to you. It can drag you in all sorts of directions and distract you from more important matters – if you allow it to. Don’t. Said testing expert Michael Bolton of DevelopSense: “Some people enslave numbers. They make numbers work too hard, and too often. I’ve seen organizations collect piles of data about defect escape ratios and defect detection percentages.
Load more

Recommended publications
  • Automated Web Application Testing Using Search Based Software Engineering
    Automated Web Application Testing Using Search Based Software Engineering Nadia Alshahwan and Mark Harman CREST Centre University College London London, UK fnadia.alshahwan.10,[email protected] Abstract—This paper introduces three related algorithms and [21]. However, of 399 research papers on SBST,1 only one a tool, SWAT, for automated web application testing using Search [20] mentions web application testing issues and none applies Based Software Testing (SBST). The algorithms significantly search based test data generation to automate web application enhance the efficiency and effectiveness of traditional search based techniques exploiting both static and dynamic analysis. The testing. combined approach yields a 54% increase in branch coverage and Popular web development languages such as PHP and a 30% reduction in test effort. Each improvement is separately Python have characteristics that pose a challenge when ap- evaluated in an empirical study on 6 real world web applications. plying search based techniques such as dynamic typing and identifying the input vector. Moreover, the unique and rich Index Terms—SBSE; Automated Test data generation; Web nature of a web application’s output can be exploited to aid applications the test generation process and potentially improve effective- ness and efficiency. This was the motivation for our work: We seek to develop a search based approach to automated I. INTRODUCTION web application testing that overcomes challenges and takes advantage of opportunities that web applications offer. The importance of automated web application testing de- rives from the increasing reliance on these systems for busi- In this paper we introduce an automated search based ness, social, organizational and governmental functions.
    [Show full text]
  • Web Gui Testing Checklist
    Web Gui Testing Checklist Wes recrystallizing her quinone congruously, phytophagous and sulphonic. How imponderable is Schroeder when barbate whileand soft-footed Brewer gliff Zachery some incisure yakety-yak affluently. some chatoyancy? Fulgurating and battiest Nealson blossoms her amontillados refine Wbox aims to the field to be able to the automated support data, testing web gui checklist Planned testing techniques, including scripted testing, exploratory testing, and user experience testing. This gui content will the css or dynamic values? Test all input fields for special characters. For instance, create test data assist the maximum and minimum values in those data field. Assisted by timing testing is not tested to the order to achieve true black art relying on gui testing web checklist will best. The web hosting environments you start all web testing gui checklist can provide tests has had made. The gui testing procedures are the weak factors causing delays in agile here offering, gui testing web? At anytime without giving us a testing web gui checklist can also has on. How gui testing checklist for a gui testing web checklist to induce further eliminating redundant if there is transmitted without the below to use of jobs with. Monkey testing tool that an application or even perform testing web gui changes some test android scripts behind successful only allows an. Discusses the preceding css or if a sql injections through an application penetration testing on gui testing web? How much regression testing is enough? Fully automated attack simulations and highly automated fuzzing tests are appropriate here, and testers might also use domain testing to pursue intuitions.
    [Show full text]
  • A Systematic Review on Regression Testing for Web-Based Applications
    Journal of Software A Systematic Review on Regression Testing for Web-Based Applications Anis Zarrad* Department of Computer Science and Information Systems, Prince Sultan University, Riyadh, Saudi Arabia. * Corresponding author. Tel.: +966114948531; email: [email protected] Manuscript submitted December 12, 2014; accepted April 16, 2015. doi: 10.17706/jsw.10.8.971-990 Abstract: Web-based applications and their underlying parts are growing rapidly to offer services over the internet around the world. Web applications play critical roles in various areas such as community development, business, academic, sciences etc. Therefore their correctness and reliability become important factors with each succeeding release. Regression testing is an important means to ensure such factors for every new released version. Regression testing is a verification process to discover the impact of changes in other interconnected modules. With the goal of selecting an appropriate regression testing approach to respond adequately to any changes in Web applications, we conduct a complete systematic study of regression testing techniques in Web applications. Out of 64, we identified a total of 22 papers reporting experiments and case studies. We present a qualitative analysis for used tools, an overview of test case section techniques and empirical study evaluation for every selected work. No approaches were found clearly superior to other since results depend on many varying factors and the deployment environments. We identified the need of evidences where approaches are evaluated cost effective rather than technical description. Key words: Regressing testing, web-based application testing, empirical study, test set, software testing. 1. Introduction In today’s scenario, as the world became global and with the advent of internet technologies, Web-based applications become more effective manner for enterprises, and academic entities to produce business strategies and policies.
    [Show full text]
  • Getting Started with Testcomplete 14 Desktop, Web, and Mobile Testing Tutorials 2
    Getting Started with TestComplete 14 Desktop, Web, and Mobile Testing Tutorials 2 About the Tutorial With TestComplete, you can test applications of three major types: desktop, web and mobile: • Desktop applications - these applications are executed on desktop computers running the Windows operating system. • Web applications - these applications are executed in web browsers (including those web browsers that are embedded into desktop applications). • Mobile applications - these applications are executed on Android or iOS devices. This document is for novice users. It provides a brief overview of automated testing and of the product, and includes tutorials that explain how to create tests for major application types. After you read these tutorials, you will be able to create, modify and execute tests for desktop, web, and mobile applications. smartbear.com TestComplete by SmartBear Software Introducing Automated Testing and TestComplete 3 Table of Contents INTRODUCING AUTOMATED TESTING AND TESTCOMPLETE ..................................................... 5 Automated Testing ......................................................................................................................................... 5 Test Types....................................................................................................................................................... 5 TestComplete Projects and Project Items ....................................................................................................... 6 TestComplete User
    [Show full text]
  • SOA Test Governance: Enabling Service Integration Testing Across Organization and Technology Borders
    Istituto di Scienza e Tecnologie dell'Informazione “A. Faedo” Software Engineering Laboratory SOA Test Governance: enabling service integration testing across organization and technology borders Antonia Bertolino Andrea Polini [email protected] [email protected] Keynote Talk @ WebTest2009: 1st Int. Wksp on Web Testing April 4th, 2009 Co-located with ICST 2009, Denver, Colorado, USA Agenda SOA SOA Governance SOA Testing Challenges SOA Test Governance (STG) STG Cases: Audition SOCT Conclusions&discussion WebTest2009: SOA Test Governance ©2009 A.Bertolino, A. Polini 2 SOA: a new paradigm ¾ Service-orientation is claimed to be the “New Computing Paradigm for the Networked World” (c.f.r. the theme of the First ICSOC in 2003) ¾ Revolution or Evolution? Service-orientation (like Rome) “was not built in a day”: it is not a revolution in IT, rather it is the natural (and relatively slow) evolution from the confluence of several recent and less recent approaches and technologies. ¾ The WWW era in the 90’s heralded SOA by emphasizing: shift from intra-organization to inter-organization integration need: “B2B” advent increasingly dynamic nature of IT market: companies enter and exit the scene continuously and fastly WebTest2009: SOA Test Governance ©2009 A.Bertolino, A. Polini 3 SOA promises ¾ IT-based companies seem infatuated of the SO wave and in different ways and with different expectations are jumping on the bandwagon. ¾ The new paradigm brings in many attracting features, among which: reuse interoperability loose coupling
    [Show full text]
  • Benchmarking Web-Testing - Selenium Versus Watir and the Choice of Programming Language and Browser
    Benchmarking Web-testing - Selenium versus Watir and the Choice of Programming Language and Browser Miikka Kuutila, M3S, ITEE, University of Oulu, Finland Mika Mäntylä, M3S, ITEE, University of Oulu, Finland Päivi Raulamo-Jurvanen, M3S, ITEE, University of Oulu, Finland Email: [email protected], Postal address: P.O.Box 8000 FI-90014 University of Oulu Abstract Context: Selenium is claimed to be the most popular software test automation tool. Past academic works have mainly neglected testing tools in favour of more methodological topics. Objective: We investigated the performance of web-testing tools, to provide empirical evidence supporting choices in software test tool selection and configuration. Method: We used 4*5 factorial design to study 20 different configurations for testing a web-store. We studied 5 programming language bindings (C#, Java, Python, and Ruby for Selenium, while Watir supports Ruby only) and 4 Google Chrome, Internet Explorer, Mozilla Firefox and Opera. Performance was measured with execution time, memory usage, length of the test scripts and stability of the tests. Results: Considering all measures the best configuration was Selenium with Python language binding for Google Chrome. Selenium with Python bindings was the best option for all browsers. The effect size of the difference between the slowest and fastest configuration was very high (Cohen’s d=41.5, 91% increase in execution time). Overall Internet Explorer was the fastest browser while having the worst results in the stability. Conclusions: We recommend benchmarking tools before adopting them. Weighting of factors, e.g. how much test stability is one willing to sacrifice for faster performance, affects the decision.
    [Show full text]
  • Mobile and Web Testing Using SOFY
    Mobile and Web Testing using SOFY Intelligent Testing, Increased Productivity Introduction to SOFY Our Mission Help customers ship quality code faster Increase Developer and Tester productivity SOFY is a testing platform for Mobile Apps and Websites that augments manual testing and automates test cases with no coding, powered by patented ML algorithms. Scope of SOFY Client-side UI Browser/ Unit/ User Functional/ Security and Accessibility Device Integration Pen Testing API Testing Acceptance UI Testing Performance Testing Testing Testing Testing Testing Cost of testing & length of time Level of Automation SOFY Confidential Today’s Testing Methodology Manual Testing Automated Testing • Repeatable scenarios • Initial/Default testing method • Scale Testing • Minimal upfront investment Use cases • Regression Testing • Costly and time consuming • Requires extensive coding, high skillset • Lacks Consistency, Traceability • Needs use of multiple tools Challenges • Not scalable for cross device, OS testing • Lacks resiliency to UI changes, device types, • Requires training for domain knowledge dynamic content • Inefficiency between Development and Testing • Failure analysis across tools is tedious SOFY’s Approach to Testing Augmented Automated Testing Autonomous Manual Testing Exploratory Testing • Create/Edit automated • Instantly Configure your test with NOCODE • Navigate the entire test environment & devices application autonomously • Run at scale across • Find visual quality, • Test and Report bugs with devices with no code accessibility,
    [Show full text]
  • Free Software Testing Tools for Web Applications
    Free Software Testing Tools For Web Applications Vlad magnetizes too-too if paragogic Jedediah kythed or assail. Bronchial and paratactical Tate bonnets her beating skunks tar and ski-jump jejunely. Munroe underpinned crankily? Jason boog has never have free tool of applications across all the testing has also available for specific needs to reduce risk. This particular web performance testing tool can be used for intranet applications and websites. In order to perform unit testing, there are some tools that can help developers. It makes web tests faster and easier to infamous, and provides abstractions to make tests more ash and reusable. It can still widely used on developer perspectives, and can start as well as well as a server or free software testing tools for applications testing expected to handle changes and solaris. Selenium IDE is an integrated development environment for Selenium scripts. To a testing software test components. Provides tracking, management, organization and reporting capabilities for software testing and test case management. Yawet offers reports, step libraries, parametrisation and custom extensions. No Credit Card Required Free Sign Up an Anytime. Understand the data flow. It environments including testability, integrates with tools offer specialized ide. UFT is a powerful UI testing tool. It allows the testers to accomplish unit and regression test without programming effort. NET by Alexandru Botez. UFT, you can run it across multiple nodes from a single controller, thus enabling hundreds of tests to be executed simultaneously. We do web applications for free tools, depending on the same environment it allows you need it uses cookies.
    [Show full text]
  • Web Performance Testing: Methodologies, Tools and Challenges
    International Journal of Scientific Engineering and Research (IJSER) www.ijser.in ISSN (Online): 2347‐3878 Volume 2 Issue 1, January 2014 Web Performance Testing: Methodologies, Tools and Challenges Vinayak Hegde1, Pallavi M S2 1Assistant .Professor, Dept. of Computer Science, Amrita Vishwa Vidyapeetham, Mysore Campus, India 2Lecturer, Department of Computer Science, Amrita Vishwa Vidyapeetham, Mysore Campus, India Abstract: The Internet is gently becoming the essential requirement for the business world, with web applications as the brains. This means that software faults in web applications have potentially disastrous consequences. Most work on web applications has been on making them more powerful, but relatively little has been done to ensure their quality. Important quality attributes for web applications include reliability, availability, interoperability and security. Web Performance testing is a type of testing intended to determine the responsiveness, reliability, throughput, interoperability and scalability of a system and /or application under a given workload. It could also be defined as a process of determining the speed or effectiveness of a computer, network, software application or device. Testing can be conducted on software applications, system resources, targeted application components, databases and a whole lot more. It normally involves an automated test suite as this allows for easy, repeatable simultaneous a variety of normal peak and exceptional load conditions. Such forms of testing help verify whether a system or application meets the specifications claimed by its vendor. This paper emphasis on methodology of performance testing and explains about various diagnostic tools to implement testing to overcome from single point of failure. This paper also explains about challenges of web performance testing and helps for one who takes up further research activity.
    [Show full text]
  • Modeling and Testing Web-Based Applications
    Modeling and Testing Web-based Applications Ye Wu and Jeff Offutt Information and Software Engineering Department George Mason University Fairfax, VA 22030, USA (+1) 703-9934-1651 / 1654 {wuye,ofut}@ise.gmu.edu Abstract The Internet is quietly becoming the body of the business world, with web applications as the brains. This means that software faults in web applications have potentially disastrous conse- quences. Most work on web applications has been on making them more powerful, but relatively little has been done to ensure their quality. Important quality attributes for web applications include reliability, availability, interoperability and security. Web applications share some characteristics of client-server, distributed, and traditional programs, however there are a number of novel aspects of web applications. These include the fact that web applications are “dynamic”, due to factors such as the frequent changes of the application requirement as well as dramatic changes of the web technologies, the fact that the roles of the clients and servers change dynamically, the heterogeneity of the hardware and software components, the extremely loose coupling and dynamic integration, and the ability of the user to directly affect the control of execution. In this paper, we first analyze the distinct features of web-based applications, and then define a generic analysis model that characterizes the typical behaviors of web-based applications indepen- dently of different technologies. Based on this analysis, a family of testing techniques is discussed to ensure different level of quality control of web applications under various situations. 1 Introduction Driven by the extreme demands of the business world and enthusiasm of the public, the internet has become indispensable to business, education, and even our personal lives.
    [Show full text]
  • Website Performance Testing
    Website Performance Testing Like any software product, your website or web application requires thorough testing before it goes live to ensure a quality user experience. Web users are accustomed to a high level of functionality, responsiveness and usability. Increasingly, they lack patience for web sites that do not provide a fulfilling experience. Thus, testing the performance aspects of your website is critical. eBook Website Performance Testing Website Performance Testing Performance testing assesses your site's efficiency compared to its specifications. Its main focus is overall responsiveness under average loads. This category of testing should be performed early and often in order to uncover imbalances in the software architecture or its implementation. Monopolization of resources, processing bottlenecks and unexpected latency are likely suspects, which are easier to correct early on rather than at later stages in development or testing. The end result of these performance tests and subsequent software changes is a set of baseline measurements you apply whenever the web site is changed to ensure that responsiveness and processing efficiency are stable and improving. Let us look at two specific types of website performance testing: Load Testing and Stress Testing. Load Testing Stress Testing After early performance test results stabilizes, Stress tests apply maximum burden on the load tests that simulate thousands or even website in order to uncover its breaking millions of users over long time periods points. Equally important, these tests measure are appropriate for measuring software how gracefully your site crashes and recovers. endurance and volume capacity. These take Success is declared when the site can fail you one step closer to real-world conditions.
    [Show full text]
  • A Brief Survey on Web Application Performance Testing Tools Literature Review
    International Journal of Latest Trends in Engineering and Technology (IJLTET) A Brief Survey on Web Application Performance Testing Tools Literature Review Isha Arora M.Tech Scholar, Department of Computer Science and Engineering , PIET, PANIPAT, INDIA Vikram Bali Department of Computer Science and Engineering, PIET, PANIPAT, INDIA Abstract - Web Testing is the complete testing of a web based system before it goes live on the internet. Due to increase in number of websites, the demand is for accurate, faster, continuous & attractive access to web content. So, before publishing it online, Testing process is required which includes basic functionalities of the site, User Interface compatibility, its accessibility, effect of traffic on server etc. This leads to the requirement of one handy and intelligent tool or application which provides these facilities of testing in an easy & automated way such that time & cost are minimized and testing is no more a headache. This paper presents an extensive Literature Survey of performance Automation Testing tools. This paper is first of two, the second is going on to consider the framework of Performance Automation tool that will carry its own new set of features which are desired by professionals and currently unavailable. Keywords: Software testing, Automation testing, Performance testing, Testing tools, Non functional testing I. INTRODUCTION Software Testing is the process used to help identify the correctness, completeness, security, and quality of developed computer software. Testing is a process of technical investigation, performed on behalf of stakeholders, that is intended to reveal quality related information about the product with respect to the actual functioning of the product, it includes the process of executing a program or application with the intent of finding errors.
    [Show full text]