DOCSLIB.ORG

Software Testing Fundamentals

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Software Testing Fundamentals Software Testing Fundamentals By Urmi Saha What is software testing? 1. Process of verifying and validating that a software or application is bug free AND 2. The software meets the technical requirements as guided by it’s design and development AND 3. The software meets the user requirements effectively and efficiently with handling all the exceptional and boundary cases Software testing is divided into 2 steps: 1. Verification This refers to the set of tasks that ensure that software correctly implements a specific function. “Are we building the product right?” 2. Validation This refers to a different set of tasks that ensure that the software that has been built is traceable to customer requirements. “Are we building the right product?” Types of software testing: 1. Manual Testing The tester takes over the role of an end-user and tests the software to identify any unexpected behavior or bug without any automated tool or script. 2. Automation Testing The tester writes scripts and uses another software to test the product. Used to re-run the test scenarios that were performed manually and repeatedly Techniques of software testing 1. Black Box Testing ● Tester doesn’t have access to the source code or the internal logical structure of the software ● Functionality of the software is not known ● This can only be done by trial and error method 2. White Box Testing ● Tester is aware of the internal workings of the product ● Tester has access to it’s source code ● Testing conducted by making sure that all internal operations are performed according to the specifications Levels of software testing 1. Unit Testing: Individual units/components of a software are tested 2. Integration Testing: Individual units are combined and tested as a group 3. System Testing: Complete, integrated system/software is tested 4. Acceptance Testing: Assess whether it is acceptable for delivery Unit Testing Recap “Unit Testing is defined as a type of software testing where individual units/ components of a software are tested” ● Performed during the development (coding) of an application ● Isolate a section of code and verify its correctness ● A unit may be an individual function or procedure ● Usually performed by the developer Unit Testing “How is it performed?” ● A developer writes a section of code in the application just to test the function. ○ ”They would later comment out and finally remove the test code when the application is deployed.” ● Relies on mock objects being created to test sections of code that are not yet part of a complete application. ○ “Mock objects fill in for the missing parts of the program” Example of a unit test case (Django) ● Django’s unit tests use a Python standard from django.test import TestCase library module: “unittest” from myapp.models import Animal class AnimalTestCase(TestCase): ● Here is an example which subclasses from django.test.TestCase, which is a subclass of def setUp(self): unittest.TestCase that runs each test inside a Animal.objects.create(name="lion", sound="roar") Animal.objects.create(name="cat", sound="meow") transaction to provide isolation => def test_animals_can_speak(self): ● When you run your tests, the default """Animals that can speak are correctly identified""" behavior of the test utility is to find all the test lion = Animal.objects.get(name="lion") cat = Animal.objects.get(name="cat") cases (that is, subclasses of self.assertEqual(lion.speak(), 'The lion says "roar"') unittest.TestCase) in any file whose name self.assertEqual(cat.speak(), 'The cat says "meow"') begins with test, automatically build a test suite out of those test cases, and run that suite. End-to-End Testing with Selenium Web App Testing Automation ● Manual tests are repeated Why automated often during development cycles for source code testing? changes. ● Once automated tests are created they can easily be repeated. What is ● Selenium automates browsers. ● It is a software testing Selenium? framework for web applications. Browser Automation ● Can be used for web-based administration tasks. 1. Python is simpler, faster and more compact than any other Why Python in programming language. 2. It has far less verbose and Selenium? easy to use. 3. Selenium Python bindings Selenium test cases can be provides a simple API to write written in Java, Python, C#, test-cases using Selenium Ruby, Perl, PHP, etc. WebDriver. 1. Gecko is a web browser engine 2. Gecko Driver is the link between your tests in Selenium Gecko Driver and the Firefox browser 3. Gecko Driver is an executable Drivers for web-driver file needed in one of the system path before starting automation tests Installation 1. Selenium: pip3 install selenium 2. Geckodriver for firefox: a. wget https://github.com/mozilla/geckodriver/releases/download/v0.20.1/geckodriver-v0.20.1-linux64. tar.gz b. tar -xvzf geckodriver-v0.20.1-linux64.tar.gz c. rm geckodriver-v0.20.1-linux64.tar.gz d. chmod +x geckodriver e. cp geckodriver /usr/local/bin/ 3. Write test script and run. Example of script import unittest Console Output from selenium import webdriver from selenium.webdriver.common.keys import Keys class PythonOrgSearch(unittest.TestCase): python test_python_org_search.py . def setUp(self): ----------------------------------------------------------------- self.driver = webdriver.Firefox() ----- Ran 1 test in 15.566s def test_search_in_python_org(self): driver = self.driver driver.get("http://www.python.org") OK self.assertIn("Python", driver.title) elem = driver.find_element_by_name("q") elem.send_keys("pycon") elem.send_keys(Keys.RETURN) assert "No results found." not in driver.page_source def tearDown(self): self.driver.close() if __name__ == "__main__": unittest.main() Examples of Selenium-WebDriver API Commands and Operations Locating UI Elements (WebElements) 1. By ID: html: <div id="coolestWidgetEvah">...</div> selenium code: element = driver.find_element_by_id("coolestWidgetEvah") 2. By Class Name: html: <div class="cheese"><span>Cheddar</span></div><div class="cheese"><span>Gouda</span></div> selenium code: cheeses = driver.find_elements_by_class_name("cheese") 3. By Name: html: <input name="cheese" type="text"/> selenium code: cheese = driver.find_element_by_name("cheese") 4. By XPath: html: <input type="text" name="example" /> selenium code: inputs = driver.find_elements_by_xpath("//input") etc. Getting text values element = driver.find_element_by_id("element_id") element.text Moving Between Windows and Frames driver.switchTo().window("windowName"); Popup Dialogs alert = driver.switch_to.alert Navigation driver.navigate().forward(); driver.navigate().back(); Reference http://selenium-python.readthedocs.io http://selenium-python.readthedocs.io/getting-started.html https://www.seleniumhq.org/docs/03_webdriver.jsp https://docs.djangoproject.com/en/dev/topics/testing/#writing-unit-tests https://docs.djangoproject.com/en/dev/topics/testing/overview/ https://docs.djangoproject.com/en/dev/topics/testing/overview/#running-tests https://www.dev2qa.com/django-unit-test-example/ Thank You.
Load more

Recommended publications
  • A Framework for Evaluating Performance of Software Testing Tools
    INTERNATIONAL JOURNAL OF SCIENTIFIC & TECHNOLOGY RESEARCH VOLUME 9, ISSUE 02, FEBRUARY 2020 ISSN 2277-8616 A Framework For Evaluating Performance Of Software Testing Tools Pramod Mathew Jacob, Prasanna Mani Abstract: Software plays a pivotal role in this technology era. Due to its increased applicable domains, quality of the software being developed is to be monitored and controlled. Software organization follows many testing methodologies to perform quality management. Testing methodologies can be either manual or automated. Automated testing tools got massive acceptability among software professionals due to its enhanced features and functionalities than that of manual testing. There are hundreds of test automation tools available, among which some perform exceptionally well. Due to the availability of large set of tools, it is a herculean task for the project manager to choose the appropriate automation tool, which is suitable for their project domain. In this paper, we derive a software testing tool selection model which evaluates the performance aspects of a script-based testing tool. Experimental evaluation proves that, it can be used to compare and evaluate various performance characteristics of commercially accepted test automation tools based on user experience as well as system performance. Index Terms: Automated testing, Software testing, Test script, Testing Tool, Test bed, Verification and Validation. —————————— ◆ —————————— 1 INTRODUCTION S OFTWARE is advanced in recent days by enhancing its applicable domains. Software is embedded in almost all electronic gadgets and systems. In this scenario the quality of the software plays a significant role. The customer or end – user should be satisfied which is primarily depended on quality and capability of the software being developed.
    [Show full text]
  • Automated Testing of Firmware Installation and Update Scenarios for Peripheral Devices
    DEGREE PROJECT IN COMPUTER SCIENCE AND ENGINEERING, SECOND CYCLE, 30 CREDITS STOCKHOLM, SWEDEN 2019 Automated testing of firmware installation and update scenarios for peripheral devices DAG REUTERSKIÖLD KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE Automated testing of firmware installation and update scenarios for peripheral devices DAG REUTERSKIÖLD Master in Computer Science Date: August 12, 2019 Supervisor: Hamid Faragardi Examiner: Elena Troubitsyna School of Electrical Engineering and Computer Science Host company: Tobii AB Swedish title: Automatisering av enhetsinstallation, uppdatering och testning med hjälp av virtuella maskiner iii Abstract This research presents an approach to transition from manual to automated testing of hardware specific firmware. The manual approach for firmware test- ing can be repetitive and time consuming. A significant proportion of the time is spent on cleaning and re-installing operating systems so that old firmware does not interfere with the newer firmware that is being tested. The approach in this research utilizes virtual machines and presents an automation framework. One component of the automation framework is an application to imitate con- nected peripheral devices to bypass hardware dependencies of firmware in- stallers. The framework also consists of automation and pipeline scripts with the objective to execute firmware installers and detect errors and abnormalities in the installation and updating processes. The framework can run on locally hosted virtual machines, but is most applicable using cloud hosted virtual ma- chines, where it is part of a continuous integration that builds, downloads, installs, updates and tests new firmware versions, in a completely automated manner. The framework is evaluated by measuring and comparing execution times with manually conducted installation and updating tests, and the result shows that the framework complete tests much faster than the manual approach.
    [Show full text]
  • GUI-Guided Test Script Repair for Mobile Apps
    This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication. The final version of record is available at http://dx.doi.org/10.1109/TSE.2020.3007664 1 GUI-Guided Test Script Repair for Mobile Apps Minxue Pan, Tongtong Xu, Yu Pei, Zhong Li, Tian Zhang, Xuandong Li Abstract—Graphical User Interface (GUI) testing is widely used to test mobile apps. As mobile apps are frequently updated and need repeated testing, to reduce the test cost, their test cases are often coded as scripts to enable automated execution using test harnesses/tools. When those mobile apps evolve, many of the test scripts, however, may become broken due to changes made to the app GUIs. While it is desirable that the broken scripts get repaired, doing it manually can be preventively expensive if the number of tests need repairing is large. We propose in this paper a novel approach named METER to repairing broken GUI test scripts automatically when mobile apps evolve. METER leverages computer vision techniques to infer GUI changes between two versions of a mobile app and uses the inferred changes to guide the repair of GUI test scripts. Since METER only relies on screenshots to repair GUI tests, it is applicable to apps targeting open or closed source mobile platforms. In experiments conducted on 22 Android apps and 6 iOS apps, repairs produced by METER helped preserve 63.7% and 38.8% of all the test actions broken by the GUI changes, respectively.
    [Show full text]
  • Test Script Language
    CHAPTER 11 TEST SCRIPT LANGUAGE We will now look at the TSL language. You have already been exposed to this language at various points of this book. All the recorded scripts that WinRunner creates when you perform an operation is in TSL code syntax. Keep in mind that while mastery of TSL is not required for creating automated tests, knowledge of the language helps to enhance the recorded tests and to create highly sophisticated tests. Skillful usage of TSL can limit the need for manual intervention when running your test. It can also make your test less error‐prone. Test Script Language TSL is the script language used by WinRunner for recording and executing scripts. In this chapter, I will provide you with the foundational concepts of TSL. Think of these as the building blocks of TSL scripting that will help you learn how to write the language. Another useful resource to review while learning TSL is the WinRunner help system. TIP: To access the WinRunner help system at any time, press the F1 key. The TSL language is very compact containing only a small number of operators and keywords. If you are proficient in any programming language, you will find TSL easy to learn. In fact, you will find it much easier than learning a programming language because TSL is a script language and as such, does not have many of the complex syntax structures you may find in programming languages. On the other hand, TSL as a script language has considerably less features and capabilities than a programming language.
    [Show full text]
  • An Introduction to Test Driven Development Using Perl
    An Introduction to Test Driven Development Using Perl Grant McLean, Catalyst IT Limited <[email protected]> September 2008 This article describes the practise of Test Driven Development and illustrates it with a worked example. The sample tests, source code and tools are written in Perl however the concepts should be applicable to any programming language. Introducing Test Driven Development Few people would dispute that testing is a critical part of any software development project. It is unfortunate however that testing is often left until the end. This may be due to a mistaken belief that tests are useless until you have something to test. Test Driven Development (TDD) positions testing in a critical role at the centre of development activities. Development models based around TDD favour a cyclic flow with a series of iterations building one upon another to deliver functionality in stages. Each iteration follows a flow similar to that illustrated in figure 1. Figure 1. An Idealised Test Driven Development cycle The process begins with requirements gathering – deciding what will be delivered and defining success criteria (how we'll know when we've delivered it). The requirements definition feeds into the development of a test plan. Development of the program code can proceed once tests have been written to specify correct behaviour for the code. When tests have been written to describe all the requirements and code has been written to pass all the tests, the development is complete and the system can be deployed. The process then begins again with the definition of the requirements for the next set of deliverables.
    [Show full text]
  • Benchmarking Web-Testing - Selenium Versus Watir and the Choice of Programming Language and Browser
    Benchmarking Web-testing - Selenium versus Watir and the Choice of Programming Language and Browser Miikka Kuutila, M3S, ITEE, University of Oulu, Finland Mika Mäntylä, M3S, ITEE, University of Oulu, Finland Päivi Raulamo-Jurvanen, M3S, ITEE, University of Oulu, Finland Email: [email protected], Postal address: P.O.Box 8000 FI-90014 University of Oulu Abstract Context: Selenium is claimed to be the most popular software test automation tool. Past academic works have mainly neglected testing tools in favour of more methodological topics. Objective: We investigated the performance of web-testing tools, to provide empirical evidence supporting choices in software test tool selection and configuration. Method: We used 4*5 factorial design to study 20 different configurations for testing a web-store. We studied 5 programming language bindings (C#, Java, Python, and Ruby for Selenium, while Watir supports Ruby only) and 4 Google Chrome, Internet Explorer, Mozilla Firefox and Opera. Performance was measured with execution time, memory usage, length of the test scripts and stability of the tests. Results: Considering all measures the best configuration was Selenium with Python language binding for Google Chrome. Selenium with Python bindings was the best option for all browsers. The effect size of the difference between the slowest and fastest configuration was very high (Cohen’s d=41.5, 91% increase in execution time). Overall Internet Explorer was the fastest browser while having the worst results in the stability. Conclusions: We recommend benchmarking tools before adopting them. Weighting of factors, e.g. how much test stability is one willing to sacrifice for faster performance, affects the decision.
    [Show full text]
  • Reducing Software Testing Time with Combinatorial Testing and Test Automation Akalanka Bandara Mailewa St
    St. Cloud State University theRepository at St. Cloud State Culminating Projects in Computer Science and Department of Computer Science and Information Information Technology Technology 6-2015 Reducing Software Testing Time with Combinatorial Testing and Test Automation Akalanka Bandara Mailewa St. Cloud State University Follow this and additional works at: https://repository.stcloudstate.edu/csit_etds Part of the Computer Sciences Commons Recommended Citation Mailewa, Akalanka Bandara, "Reducing Software Testing Time with Combinatorial Testing and Test Automation" (2015). Culminating Projects in Computer Science and Information Technology. 3. https://repository.stcloudstate.edu/csit_etds/3 This Starred Paper is brought to you for free and open access by the Department of Computer Science and Information Technology at theRepository at St. Cloud State. It has been accepted for inclusion in Culminating Projects in Computer Science and Information Technology by an authorized administrator of theRepository at St. Cloud State. For more information, please contact [email protected]. Reducing Software Testing Time with Combinatorial Testing and Test Automation by Akalanka Bandara Mailewa A Starred Paper Submitted to the Graduate Faculty of St. Cloud State University in Partial Fulfillment of the Requirements for the Degree Master of Science in Computer Science June, 2015 Starred Paper Committee: Jayantha Herath, Chairperson Andrew Anda Susantha Herath 1 2 Abstract The development of large software systems is a complex and error prone process. Errors may occur at any stage of software development. These errors, sometimes referred to as bugs, can cause great losses in terms of both time and money if not identified and removed as early as possible. Testing a software product is costly, since it takes much time and need to test many combinations of its functions, integrity, performance etc., which can be called as test cases.
    [Show full text]
  • Got Python? Unlock the Future of Test Automation Quickly
    WHITE PAPER Got Python? Unlock the Future of Test Automation Quickly Move to advanced test automation techniques using Python scripts Test automation engineers classically automate tests using shell scripts and homegrown tools. Today, test teams can more easily and robustly automate their Powerful Python tools test plans using Python. This paper covers three programming language integration and libraries: topics, so you can take your first steps into the world of automated testing. • Pytest: An extremely versatile testing framework for Python 1. Understanding Python Strengths • NumPy: A library providing 2. Python Programming Challenges multi-dimensional arrays and many high-level math 3. Testing Using Python functions • SciPy: A library to extend the capabilities of NumPy with many algorithms Introduction to Python included Developing a successful test plan depends on identifying the right automation • Mathplotlib: A library for numerical plotting and tools for the project. These tools are critical for timely and efficient electronic useful data analytics tools product development. • Pythonnet: Package for integration between Python Test automation improves efficiency in three ways. First, automation replaces and the .NET CLR manual human repetition, resulting in fewer opportunities for error. Second, automation accelerates sequential testing by automatically switching through an entire battery of tests. This is far more efficient than manual testing. Finally, automated data capture and consistent execution increase accuracy. Find us at www.keysight.com Page 1 Python is a powerful tool for writing scripts that can be used to automate testing. Python is a runtime language, easy to learn, but lacking the robust GUI and driver development support needed for full test automation.
    [Show full text]
  • Choosing a Load Testing Strategy Why and How to Optimize Application Performance
    Choosing A Load Testing Strategy Why and How to Optimize Application Performance Executive Summary Poor software quality carries with it tremendous costs to an organization. Today, virtually every business depends on software for the development, production, distribution and/or after-sales support of products and services. According to a 2002 study conducted by the United States Department of Commerce, National Institute of Standards and Technology (NIST), poor software quality in the United States, alone, results in a cost to the U.S. economy of $60 billion per year [1]. This paper introduces load testing, an important component in optimizing software quality. Proper load testing can help to mitigate the costs of poor quality. We will explore the importance of load testing, when in the software development process to load test and how optimal performance can be achieved by proper load testing. In addition to discussing the various strategies for implementing load testing within an organization, this paper will explore the very real benefits load testing returns to the organization. Finally, this paper explains the proper strategy for realizing the full benefits load testing can provide - including best practices, such as how to plan for optimal performance, the suggestion that QA should really be viewed as Production Engineering and how to extend performance assurance into production. What Is Load Testing? Load testing is the systematic exposure of an application to real world, expected usage conditions in order to predict system behavior and to pinpoint/diagnose errors in an application and its infrastructure before it is deployed. Load testing is used to analyze the following three aspects of an application’s quality of service: • Performance (response times) • Scalability (throughput) • Reliability (availability and functional integrity) There are many types of load testing, each of which performs a given function.
    [Show full text]
  • Using Tcl for Test Automation of a Large Fiber-Optic Switching System
    Using Tcl for Test Automation of a Large Fiber-Optic Switching System Hugh Dunne Ciena Corporation 10480 Ridgeview Court Cupertino CA 95014 [email protected] ABSTRACT • TL1 We describe a Tcl-based test automation system for fiber-optic • Proprietary command line interfaces for debugging telecommunications equipment. • HTTP Keywords Test Automation, SONET, CORBA, TL1, TclBlend, Expect • Proprietary XML-like interface 1. INTRODUCTION • GUI client written in Java Ciena Corporation's Core Director™ is an intelligent, high- • NBI (North-Bound Interface) - an IDL-based network performance optical networking core switch. It offers state-of- management system the-art capabilities for real-time provisioning and grooming of fiber-optic networks, supports a wide range of optical interfaces, When a Core Director powers up, it uses discovery protocols to replaces the functionality of diverse legacy equipment, and exchange information with other nodes and form an internal map dramatically reduces the cost of deploying, operating, and scaling of the network. This allows sophisticated route optimization and optical networks. protection policies to be implemented automatically. Physically, the Core Director occupies a standard 2. TESTING THE SYSTEM telecommunications equipment bay and can accommodate up to Testing this system presents a number of challenges. It must pass 32 line module cards, each of which can hold up to 8 optical a large suite of tests to demonstrate standards compliance. It modules. Each optical module has a transmit and a receive port supports a rich set of interfaces, all of which must be exercised. where data is converted between optical and electronic format. A In addition, the test automation infrastructure must interface with smaller version, the Core Director CI, is roughly half the size and a large database of test cases, and with various pieces of test hosts fewer optical modules, but has the same management equipment which are used to generate traffic to the system under interfaces: test and to simulate error conditions.
    [Show full text]
  • Mapping C++ Data Types Into a Test Specification Language
    ESPOO 2004 VTT PUBLICATIONS 542 VTT PUBLICATIONS VTT PUBLICATIONS 542 522 Jokinen, Tommi. Novel ways of using Nd:YAG laser for welding thick section austenitic stainless steel. 2004. 120 p. + app. 12 p. 523 Soininen, Juha-Pekka. Architecture design methods for application domain-specific in- tegrated computer systems. 2004. 118 p. + app. 51 p. 524 Tolvanen, Merja. Mass balance determination for trace elements at coal-, peat- and bark- fired power plants. 2004. 139 p. + app. 90 p. Pekka Pulkkinen 525 Mäntyniemi, Annukka, Pikkarainen, Minna & Taulavuori, Anne. A Framework for Off- The-Shelf Software Component Development and Maintenance Processes. 2004. 127 p. 526 Jäälinoja, Juho. Requirements implementation in embedded software development. Language Mapping C++ Data Types into a Test Specification Mapping C++ Data Types into a Test 2004. 82 p. + app. 7 p. 527 Reiman, Teemu & Oedewald, Pia. Kunnossapidon organisaatiokulttuuri. Tapaustutkimus Specification Language Olkiluodon ydinvoimalaitoksessa. 2004. 62 s. + liitt. 8 s. 528 Heikkinen, Veli. Tunable laser module for fibre optic communications. 2004. 172 p. + app. 11 p. 529 Aikio, Janne K. Extremely short external cavity (ESEC) laser devices. Wavelength tuning and related optical characteristics. 2004. 162 p. 530 FUSION Yearbook. Association Euratom-Tekes. Annual Report 2003. Ed. by Seppo Kart- tunen & Karin Rantamäki. 2004. 127 p. + app. 10 p. 531 Toivonen, Aki. Stress corrosion crack growth rate measurement in high temperature water using small precracked bend specimens. 2004. 206 p. + app. 9 p. 532 Moilanen, Pekka. Pneumatic servo-controlled material testing device capable of oper- ating at high temperature water and irradiation conditions. 2004. 154 p. 534 Kallio, Päivi. Emergence of Wireless Services.
    [Show full text]
  • Essentials of Spirent Testcenter GUI to Script Automation
    Essentials of Spirent TestCenter GUI to Script Automation Introduction 1 Introduction Course Description The Essentials of Spirent TestCenter GUI to Script Automation Web-based course is designed to present an overview of the features and functions of the Save As Script menu item contained within the Spirent TestCenter GUI. Since it is a web-based course, you can choose to view only the topics of interest or the entire course. Viewing all the topics within the course will take just under 30 minutes. Spirent Communications 2 Introduction Course Objectives After completing this course, participants will be able to: Generate and run TCL, Ruby, and Perl scripts created using the Save As Script feature. Identify and describe the key elements contained with the files generated from the Save As Script feature. Describe the purpose of the Save As Script Additional Processing Options. Download and install HLTAPI. Generate HLTAPI scripts using the Save As Script feature. Spirent Communications 3 Course Topics Overview Spirent TestCenter GUI to Script Automation Overview GUI to Script Content Additional Processing Options HLTAPI Spirent Communications 4 Essentials of Spirent TestCenter GUI to Script Automation GUI to Script Automation Overview 5 Course Topics Overview Spirent TestCenter GUI to Script Automation Overview GUI to Script Content Additional Processing Options HLTAPI Spirent Communications 6 Spirent TestCenter GUI to Script Automation Overview GUI to Script Overview The GUI to Script option enables automation on Spirent TestCenter without the typical automation overhead. Simply configure the test within the GUI. With the click of a button, generate the Test Script Then execute the Test Script from a Shell.
    [Show full text]