DOCSLIB.ORG

Testrig: a Platform Independent System Testing Tool. Candidate: Kaul Vaibhav Advisor: Shasha Dennis

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Testrig: a Platform Independent System Testing Tool. Candidate: Kaul Vaibhav Advisor: Shasha Dennis TestRig: A Platform independent system testing tool. Candidate: Kaul Vaibhav Advisor: Shasha Dennis Abstract The goal of the TestRig software is to give a test engineer a fixed interface to help him with system/integration testing of software systems. TestRig is platform independent and can be utilized to test software systems coded with any programming language. In addition to doing that, it provides templates and examples of using various Open Source testing tools to help a user design their test cases. TestRig has been designed keeping in mind the current scenario in software development where complex systems are often created using multiple programming languages across different platforms. The challenge is to have a defined set of rules that are able to test any such system. The software makes use of various open source testing tools to run tests and verify results, which enables a user to test a system at different levels such as Performance Testing, Blackbox Testing, and User Acceptance Testing. TestRig is open source and utilizes a programmer’s creativity to test across multiple scenarios. The thesis will show how different software systems have been tested using TestRig. [NYU COURANT INTITUTE OF MATHEMATICS] [TestRig] [A Platform Independent System Testing Tool] Vaibhav Kaul N11167470 [Fall 2010] A thesis submitted in partial fulfillment Of the requirements for the degree of Master of Science in the Department of Computer Science, New York University Approved: ________________________________ (Dennis Shasha, Research Advisor) Approved: ________________________________ (Lakshminarayanan Subramanian, 2nd Reader) 1 Acknowledgements I would like to thank Professor Dennis Shasha for his continued support and guidance. I am also grateful to Arthur Meacham, PhD candidate in CS for his help with understanding the Outsafe Project and for his inputs on testing it. A special thanks to Professor Lakshminarayanan Subramanian for taking the time to read the thesis text and for his feedback. Research and Development TestRig is a platform independent System testing tool developed over Fall 2010. It started out as an application to test the Outsafe application in a cloud based platform but has been extended to incorporate other platforms as well. 2 Contents Introduction .......................................................................................................................... 5 Background ........................................................................................................................... 6 The Software Development Lifecycle ................................................................................................................................. 6 Traditional Waterfall Model....................................................................................................................................................... 6 Iterative Development .................................................................................................................................................................. 7 Software Testing ........................................................................................................................................................................ 8 Approach ............................................................................................................................................................................................ 8 Black Box Testing ................................................................................................................................................................................. 8 White Box Testing ................................................................................................................................................................................ 8 Types of Testing ............................................................................................................................................................................... 8 Functional Testing ............................................................................................................................................................................... 8 Non-Functional Testing ..................................................................................................................................................................... 9 Automation Testing ....................................................................................................................................................................... 9 Test Driven Development (TDD) ................................................................................................................................................. 10 Automation Testing Tools [2] ....................................................................................................................................................... 11 Automation Testing Frameworks ............................................................................................................................................... 12 TestRig ................................................................................................................................ 13 Goal .............................................................................................................................................................................................. 13 State of the Art ......................................................................................................................................................................... 14 The TestRig Operation Contract ............................................................................................................................................ 14 What sets it apart .................................................................................................................................................................... 16 Examples ............................................................................................................................. 18 Test whether a number is prime or not ........................................................................................................................... 18 Context ............................................................................................................................................................................................. 18 Tools Used ....................................................................................................................................................................................... 18 Operation Contract Implementation ................................................................................................................................... 19 JUnit Test Case ......................................................................................................................................................................... 19 Context ............................................................................................................................................................................................. 19 Tools Used ....................................................................................................................................................................................... 20 Operation Contract Implementation ................................................................................................................................... 20 UI Testing in Windows (C#) .................................................................................................................................................. 21 Context ............................................................................................................................................................................................. 21 Tools Used ....................................................................................................................................................................................... 21 Selenium ................................................................................................................................................................................................. 21 How it Works .................................................................................................................................................................................. 21 Sending the Facebook Message .................................................................................................................................................... 26 Performance Test For WCF Web Service ......................................................................................................................... 36 Context ............................................................................................................................................................................................. 36 Tools Used ....................................................................................................................................................................................... 36 3 Microsoft WCF [6] .............................................................................................................................................................................
Load more

Recommended publications
  • Dr. C#: a Pedagogic IDE for C# Featuring a Read-Eval-Print-Loop by Dennis Lu
    RICE UNIVERSITY Dr. C#: A Pedagogic IDE for C# Featuring a Read-Eval-Print-Loop by Dennis Lu ATHESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE Master of Science APPROVED,THESIS COMMITTEE: Robert Cartwright, Chair Professor of Computer Science Joe Warren Professor of Computer Science Dan Wallach Assistant Professor of Computer Science Dung X. Nguyen Lecturer, Computer Science Houston, Texas April, 2003 Dr. C#: A Pedagogic IDE for C# Featuring a Read-Eval-Print-Loop Dennis Lu Abstract As the primary programming language of the Microsoft .NET platform, C# will play a significant role in software development for the foreseeable future. As the language rapidly gains popularity in industry, tools made for C# development fo- cus on the professional programmer, while leaving the beginning computer science student behind. To address this problem, we introduce Dr. C#, a simple lightweight develop- ment environment with an integrated, interactive Read-Eval-Print-Loop (REPL). Dr. C# helps flatten the learning curve of both the environment and the language, enabling students to quickly learn key elements of the language and focus more easily on concepts. Dr. C# thus serves not only as a learning tool for beginner students but also as a teaching tool for instructors. The editor is based on an open source IDE called SharpDevelop. This thesis describes the implementation of Dr. C# focusing primarily on building the REPL and integrating with SharpDevelop. Acknowledgments I would like to thank my advisor Professor Robert “Corky” Cartwright for giving me the opportunity and autonomy to develop Dr. C#. I would also like to thank Professor Dung Nguyen for acting as co-advisor and for his years of guidance and support.
    [Show full text]
  • Introduction to Unit Testing
    INTRODUCTION TO UNIT TESTING In C#, you can think of a unit as a method. You thus write a unit test by writing something that tests a method. For example, let’s say that we had the aforementioned Calculator class and that it contained an Add(int, int) method. Let’s say that you want to write some code to test that method. public class CalculatorTester { public void TestAdd() { var calculator = new Calculator(); if (calculator.Add(2, 2) == 4) Console.WriteLine("Success"); else Console.WriteLine("Failure"); } } Let’s take a look now at what some code written for an actual unit test framework (MS Test) looks like. [TestClass] public class CalculatorTests { [TestMethod] public void TestMethod1() { var calculator = new Calculator(); Assert.AreEqual(4, calculator.Add(2, 2)); } } Notice the attributes, TestClass and TestMethod. Those exist simply to tell the unit test framework to pay attention to them when executing the unit test suite. When you want to get results, you invoke the unit test runner, and it executes all methods decorated like this, compiling the results into a visually pleasing report that you can view. Let’s take a look at your top 3 unit test framework options for C#. MSTest/Visual Studio MSTest was actually the name of a command line tool for executing tests, MSTest ships with Visual Studio, so you have it right out of the box, in your IDE, without doing anything. With MSTest, getting that setup is as easy as File->New Project. Then, when you write a test, you can right click on it and execute, having your result displayed in the IDE One of the most frequent knocks on MSTest is that of performance.
    [Show full text]
  • Comparative Analysis of Junit and Testng Framework
    International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 05 Issue: 05 | May-2018 www.irjet.net p-ISSN: 2395-0072 Comparative Analysis of JUnit and TestNG framework Manasi Patil1, Mona Deshmukh2 Student, Dept. of MCA, VES Institute of Technology, Maharashtra, India1 Professor, Dept. of MCA, VES Institute of Technology, Maharashtra, India2 ---------------------------------------------------------------------***--------------------------------------------------------------------- Abstract - Testing is an important phase in SDLC, Testing It is flexible than Junit and supports parametrization, parallel can be manual or automated. Nowadays Automation testing is execution and data driven testing. widely used to find defects and to ensure the correctness and The Table – 1 compares different functionalities of TestNG completeness of the software. Open source framework can be and JUnit framework. used for automation testing such as Robotframework, Junit, Spock, NUnit, TestNG, Jasmin, Mocha etc. This paper compares Table -1: Functionality TestNG vs JUnit JUnit and TestNG based on their features and functionalities. Key Words: JUnit, TestNG, Automation Framework, Functionality TestNG JUnit Automation Testing, Selenium TestNG, Selenium JUnit Yes 1. INTRODUCTION Support Annotations Yes Yes Support Test Suit Initialization Yes Testing a software manually is tedious work, and takes lot of No time and efforts. Automation saves lot of time and money, Support for Tests groups Yes also it increases the test coverage
    [Show full text]
  • Test Code Adaptation Plugin for Eclipse
    TAPE Test Code Adaptation Plugin for Eclipse Lehmia Kiran Dr. Fakhar Lodhi Wafa Basit Department of Computer Sciences Department of Computer Sciences Department of Computer Sciences NUCES-FAST NUCES-FAST NUCES-FAST Lahore, Pakistan Lahore, Pakistan Lahore, Pakistan [email protected] [email protected] [email protected] Abstract — Refactoring improves the design of software and Extreme Programming (XP) [14] allows the code to change makes it easier to maintain by eliminating code smells. Manual often; consequently unit tests must also be changed. In refactoring is often error-prone and time-consuming. traditional testing, it is a tester who writes the tests but in Refactoring tools available in almost all major object oriented XP it is developers (not testers) who write tests for every programming languages like java, C++, C# etc that provide class they develop. Whenever refactoring is applied on varying degree of automation. The problem with these tools is that they do not support adaptation of unit tests of the code, many test cases fail due to inconsistency with source refactored code resulting in inconsistent unit tests. Unit tests code. The significant amount of developers’ effort is are the only safety net available to developers to verify the required to manually manipulate the testing suite on every system behavior after refactoring. Once these tests get broken single refactoring [15].The process of refactoring is there is no way to describe whether the system preserved its automated by many tools [17, 20, 21, 22]. Eclipse supports behavior or not. In this paper we provide technical details of automated refactoring and provides separate API to perform TAPE (Test code Adaptation Plug-in for Eclipse).
    [Show full text]
  • Martin Koníček Debugger Frontent for the Sharpdevelop
    Charles University in Prague Faculty of Mathematics and Physics MASTER THESIS Martin Koníček Debugger Frontent for the SharpDevelop IDE Department of Software Engineering Supervisor of the master thesis: Mgr. Pavel Ježek Study programme: Computer science Specialization: Software systems Prague 2011 I would like to thank my supervisor Pavel Ježek for his time and his very helpful, constructive advice on this thesis. I would like to extend a very honest “thank you” to David Srbecký, the author of the SharpDevelop Debugger and Daniel Grunwald, the lead developer of SharpDevelop IDE. They were both very helpful and the work on this thesis probably wouldn’t start without them. I would also like to thank Christoph Wille for running the project, Siegfried Pammer and Matt Ward for all their contributions and enthusiasm and of course Mike Krüger for originally starting the SharpDevelop project. They all deserve much respect for their work. Last but not least, I thank Google for supporting students to work on interesting open source projects. I declare that I carried out this master thesis independently, and only with the cited sources, literature and other professional sources. I understand that my work relates to the rights and obligations under the Act No. 121/2000 Coll., the Copyright Act, as amended, in particular the fact that the Charles University in Prague has the right to conclude a license agreement on the use of this work as a school work pursuant to Section 60 paragraph 1 of the Copyright Act. In Prague Martin Koníček Název práce: Debugger Frontent for the SharpDevelop IDE Autor: Martin Koníček Katedra / Ústav: Department of Software Engineering Vedoucí diplomové práce: Mgr.
    [Show full text]
  • Development Testing Platform Engines for .NET User’S Guide Version 10.3
    Development Testing Platform Engines for .NET User’s Guide Version 10.3 Parasoft Corporation 101 E. Huntington Drive, 2nd Floor Monrovia, CA 91016 Phone: (888) 305-0041 Fax: (626) 305-9048 E-mail: [email protected] URL: www.parasoft.com PARASOFT END USER LICENSE AGREEMENT PLEASE READ THIS END USER LICENSE AGREEMENT ("AGREEMENT") CAREFULLY BEFORE USING THE SOFTWARE. PARASOFT CORPORATION ("PARASOFT") IS WILLING TO LICENSE THE SOFTWARE TO YOU, AS AN INDIVIDUAL OR COMPANY THAT WILL BE USING THE SOFT- WARE ("YOU" OR "YOUR") ONLY ON THE CONDITION THAT YOU ACCEPT ALL OF THE TERMS OF THIS AGREEMENT. THIS IS A LEGALLY ENFORCEABLE CONTRACT BETWEEN YOU AND PARASOFT. BY CLICKING THE "ACCEPT" OR "YES" BUTTON, OR OTHERWISE INDICATING ASSENT ELECTRONICALLY, OR BY INSTALLING THE SOFTWARE, YOU AGREE TO THE TERMS AND CONDITIONS OF THIS AGREEMENT AND ALSO AGREE THAT IS IT ENFORCEABLE LIKE ANY WRITTEN AND NEGOTIATED AGREEMENT SIGNED BY YOU. IF YOU DO NOT AGREE TO THESE TERMS AND CONDITIONS, CLICK THE "I DO NOT ACCEPT" OR "NO" BUTTON AND MAKE NO FURTHER USE OF THE SOFTWARE. 1. DEFINITIONS 1.1. “Concurrent User" means a person that has accessed the Software at any given point in time, either directly or through an application. 1.2. "Instance" means a single occurrence of initialization or execution of software on one machine. 1.3. "Licensed Capacity" means the capacity-based license pricing metrics identified in the Order Instrument, including, without limitation, Concurrent Users, Node Locked machines, and Instances. 1.4. "Maintenance" means the maintenance and technical support services for the Software iden- tified in the Order Instrument and provided by Parasoft pursuant to this Agreement.
    [Show full text]
  • Testng Quick Guide
    TTEESSTTNNGG -- QQUUIICCKK GGUUIIDDEE http://www.tutorialspoint.com/testng/testng_quick_guide.htm Copyright © tutorialspoint.com Testing is the process of checking the functionality of the application whether it is working as per requirements and to ensure that at developer level, unit testing comes into picture. Unit testing is the testing of single entity classormethod. Unit testing is very essential to every software company to give a quality product to their customers. JUnit has driven developers to understand the usefulness of tests, especially of unit tests when compared to any other testing framework. Leveraging a rather simple, pragmatic, and strict architecture, JUnit has been able to "infect" great number of developers. Features of JUnit can be seen in Junit Features. Some of the short comings of JUnit are: Initially designed to enable unit testing only, now used for all kinds of testing. Cannot do dependency testing. Poor configuration control setUp/tearDown. Intrusive forcesyoutoextendclassesandnameyourmethodsacertainway. Static programming model forcesyoutorecompileunnecessarily. The management of different suites of tests in complex projects can be very tricky.. What is TestNG? Definition of TestNG as per its documentation is: TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use. TestNG is an open source automated testing framework; where NG of TestNG means Next Generation. TestNG is similar to JUnit especiallyJUnit4, but its not a JUnit extension. Its inspired by JUnit. It is designed to be better than JUnit, especially when testing integrated classes. The creator of TestNG is Cedric Beust. TestNG eliminates most of the limitations of the older framework and gives the developer the ability to write more flexible and powerful tests.
    [Show full text]
  • The Gallio Book
    The Gallio Book The Gallio Documentation Team The Gallio Book by The Gallio Documentation Team 3.1 beta Copyright © 2010 Gallio Project Table of Contents Preface ............................................................................................................................ vi The Aim Of This Book .............................................................................................. vi Who Should Read This Book? ..................................................................................... vi How To Read This Book ........................................................................................... vii Structure Of The Book ............................................................................................. viii This Book Is Free ...................................................................................................... ix What Is MbUnit? ....................................................................................................... ix What Is Gallio? .......................................................................................................... x The History Of Gallio & MbUnit .................................................................................. x What’s New In Gallio & MbUnit? ............................................................................. xi I. Using MbUnit and Gallio ................................................................................................. 1 1. Installing & Configuring Gallio ................................................................................
    [Show full text]
  • The Year of Xamarin.Forms
    The Year of Xamarin.Forms Aleksander Piotrowski @pelotasplus Agenda 1. What platform needs … to be a rockstar platform 2. And then ... Microsoft happened 3. What’s the difference between Xamarin and Xamarin.Forms 4. What Xamarin.Forms … cannot do (easily) 5. What Xamarin.Forms … can do 6. Apps using Xamarin(.Forms) What platform needs … to be a rockstar platform Programing Language IDE Blogs/podcasts Jake Wharton Visionary Documentation Testing tools/services Emulator Events Third-party components Library Programing Language C# F# XAML IDE IDE Jake Wharton ● Paul C Betts ○ ModernHttpClient ○ refit ○ OkHttp for Xamarin ● James Montemagno ○ MeetupManager ○ Connectivity Plugin for Xamarin ■ ~340k downloads ○ podcasts, blog posts, youtube videos Blogs/podcasts ● Fragmented Merge Conflict ● Android Developers Backstage Xamarin Podcast ● Channel 9 ● .NET rocks! ● Xamarin University paid ● blog.xamarin.com ● hardcore .Net fans also blogging about Xamarin Visionary ● never received a degree ● free software developer since 1992 ● Midnight Commander ● early Wine contributor ● worked on Linux Sun SPARC ● later on Linux for SGI Indy ● almost ported Internet Explorer to SPARC ● started GNOME project and created Gnumeric ● started Mono project to implement .NET on Linux ● Xamarin so Mono on mobile* ● Microsoft, Visual Studio for Mac, reboot mobile for Microsoft Documentation ● for all targeted platforms ● Xamarin.Android, Xamarin.iOS, Xamarin.Forms, Xamarin.everything ● Android developer could learn iOS app architecture ● explains not only C# or Xamarin-related
    [Show full text]
  • EA MDA Tutorial
    TUTORIAL: Model Driven Architecture with Enterprise Architect, MDA In Practice TUTORIAL Implementing Model Driven Architecture Using Enterprise Architect MDA in Practice By Frank Truyen [email protected] Copyright ©2006-2010, Cephas Consulting Corp. All rights reserved. Page 1 TUTORIAL: Model Driven Architecture with Enterprise Architect, MDA In Practice Cephas Consulting Corp. Implementing Model Driven Architecture Using Enterprise Architect MDA in Practice January 2010 Copyright ©2006-2010, Cephas Consulting Corp. All rights reserved. Page 2 TUTORIAL: Model Driven Architecture with Enterprise Architect, MDA In Practice Table of Contents Overview 5 Trademarks 5 Prerequisites 5 Prerequisite Paper 1, Fast Guide 5 Prerequisite Paper 2, Features 5 Background Prerequisites 5 The Enterprise Architect solution for applying MDA 6 The Tool 6 The Company Behind the Tool 6 A look under the covers 7 Creating and Modifying Transformation Templates 7 Building and leveraging UML profiles 9 Creating MOF models 11 Defining constraints in OCL 12 MDA by example 14 Base Platform Independent Model (PIM) 14 Configuration options 15 Capitalize attribute names for properties 15 Define target language collection mappings 16 Define the default database 16 Selecting the transformation template(s) 17 Running the transformation(s) 18 The target C# model 20 Generating the C# implementation code 20 The target Java model 22 Generating the Java implementation code 22 Generating an EJB entity model 24 Generating a data model 25 Copyright ©2006-2010, Cephas Consulting Corp. All rights reserved. Page 3 TUTORIAL: Model Driven Architecture with Enterprise Architect, MDA In Practice Generating an XML schema model 28 Generating a WSDL model 30 Generating a Unit Test model 34 In Conclusion 35 Additional information 36 About SPARX Systems 37 About Cephas Consulting Corp.
    [Show full text]
  • Gallio: Crafting a Toolchain
    Gallio: Crafting a Toolchain Jeff Brown [email protected] About Me Jeff Brown Lead Software Engineer at Yellowpages.com Creator of Gallio Open Source Project Lead of MbUnit Open Source Project Coding is fun! Outline Gallio and MbUnit Demo What is a Toolchain? Implementation Challenges Under the Hood Roadmap Questions Gallio Gallio is a neutral test automation platform . – Open Source. (Apache License) – Microsoft .Net platform. – Aims to provide great tools integration for many different test frameworks. – Started in October 2007 as a spinoff from MbUnit. Current release: v3.0.5. Website: www.gallio.org Gallio Vision: Gallio will be the foundation for a rich suite of interoperable tools. Test frameworks. Test runners. Test case managers. Test generators. Test reports and analytics. Test editors and IDE integration. Continuous integration facilities. Gallio Lingua franca for test tools. – Common object model. – Support for many different workflows. – Extensible. – Evolving. – Owned by the community. Objective: To unite, not to control. Gallio Tools support – Frameworks: CSUnit, MbUnit v2, MbUnit v3, MSTest, NBehave, NUnit, xUnit.net – Runners: GUI (Icarus), Command-line (Echo), TestDriven.Net, ReSharper, Visual Studio Test Tools, MSBuild, NAnt, PowerShell. – Other: CruiseControl.Net, TeamCity, TypeMock, NCover, Pex, AutoCAD 3rd party Contributions: DXCore runner (RedGreen), MSpec, and more… Gallio Trivia – Original code name provided by Andrew Stopford was to be “Galileo” but it was corrupted to “Gallio” due to a misspelling in an early email thread. MbUnit MbUnit is a test automation framework . – Open Source. (Apache License) – Aims to provide a powerful framework for unit testing and integration testing for .Net. – Started by Jonathan “Peli” de Halleux in 2004.
    [Show full text]
  • Pragmatic Unit Testing in Csharp with Nunit 2Nd
    What readers are saying about Pragmatic Unit Testing in C#... “As part of the Mono project, we routinely create and maintain extensive unit tests for our class libraries. This book is a fantastic introduction for those interested in creating solid code.” Miguel de Icaza, Mono Project, Novell, Inc. “Andy and Dave have created an excellent, practical and (of course) very pragmatic guide to unit-testing, illustrated with plenty of examples using the latest version of NUnit.” Charlie Poole, NUnit framework developer “Anybody coding in .NET or, for that matter, any language, would do well to have a copy of this book, not just on their bookshelf, but sitting open in front of their monitor. Unit testing is an essential part of any programmer’s skill set, and Andy and Dave have written (yet another) essential book on the topic.” Justin Gehtland, Founder, Relevance LLC “The Pragmatic Programmers have done it again with this highly useful guide. Aimed directly at C# programmers using the most popular unit-testing package for the language, it goes beyond the basics to show what you should test and how you should test it. Recommended for all .NET developers.” Mike Gunderloy, Contributing Editor, ADT Magazine “Using the approaches described by Dave and Andy you can reduce greatly the number of defects you put into your code. The result will be faster development of better programs. Try these techniques—they will work for you!” Ron Jeffries, www.XProgramming.com Pragmatic Unit Testing in C# with NUnit, Second Edition Andy Hunt Dave Thomas with Matt Hargett The Pragmatic Bookshelf Raleigh, North Carolina Dallas, Texas Pragmatic Bookshelf Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks.
    [Show full text]