MS (Computer Science) Thesis title:

Formal Automated Testing of an Information Management System

Submitted by:

Rozina Kamil

Roll no. 11

Reg. no: IU15M2LA011

Supervised by:

Dr. Nadeem Akhtar

i Title Formal Automated Testing of an Information Management System

By

Rozina Kamil

Roll no. 11

Reg. no: IU15M2LA011

Thesis submitted for the partial fulfilment of the requirement for the degree of

MASTER OF SCIENCE

In

COMPUTER SCIENCE

Department of Computer Science & IT The Islamic University of Bahawalpur - PAKISTAN

Fall 2015-17

i DECLARATION

Formal Automated Testing of an Information management System published source (except the references, standard mathematical or geometrical models/equations /formulae /protocols etc.). I further declare that this work has not been submitted for award of any other diploma/degree. The university may take action if information provided is found inaccurate at any stage. (In case of default, the scholar will be proceeded against as per HEC plagiarism policy).

Rozina Kamil

Roll no. 11

Reg no. IU15M2LA011

ii To,

The Controller of Examinations

The Islamia University of Bahawalpur, Pakistan

We, the supervisory committee, certify that the contents and format of thesis titled Formal Automated testing of an information management system submitted by Rozina Kamil, Roll no. 11, and Registration no. IU15M2LA011 have been found satisfactory and recommend that it be processed for evaluation by the External Examiner(s) for the award of degree.

Supervisor

Dr. Nadeem Akhtar

Department of Computer Science & IT

The Islamia University of Bahawalpur

Pakistan

iii Dedication

I dedicate my dissertation work to my all family members, friends, class mates and my supervisor. It cannot be possible for their sincere support and encourage. I would specially like to thank my supervisor Dr. Nadeem Akhtar who teaches me that:

Do not ever give in.

Do not ever stop trying

And if find yourself

Succumbing to one of the

Above for a brief moment,

Pick yourself up

Whisper a prayer and start

Where you left off

iv ACKNOWLEDGEMENTS

In the name of Allah Almighty, the Most Merciful and the Most Beneficent, who is the entire source of all knowledge and with His blessings, I enable to complete this research to complete this research successfully.

I would like to extend my gratitude to The Islamia University of Bahawalpur for providing me such environment and sources which enabled me to complete this research successfully.

I am very grateful to my Supervisor, Dr. Nadeem Akhtar, and The Islamia University of Bahawalpur for guidance as well as encouragement and material support for this research work.

At the end, my deepest appreciation belongs to my family members whose sacrifices; moral as well as financial support, encouragement and prayers contributed a lot to my success.

friends I shall answer in the words of Sir Winston Churc

Rozina Kamil

v Table of Contents Chapte 1 Introduction ...... 1

1.1Software Testing: ...... 1 1.2Types of Testing:...... 1 1.2.1Unit Testing:...... 2 1.2.2System/integration Testing:...... 2 1.2.3White Box Testing:...... 2 1.2.4Black Box Testing: ...... 3 1.2.5Grey Box Testing: ...... 4 1.3Disadvantages of over Automated Testing:...... 4 1.4Selenium Automated Testing Tool For Web Based Application:...... 7 1.5Selenium Integrated Development Environment:...... 7 1.5.1Selenium IDE: ...... 7 1.5.2Web Driver of :...... 8 1.5.3Selenium Remote Control: ...... 8 1.5.4Selenium Grid:...... 8 1.6Thesis Overview:...... 8 Chapter 2: Statement on Problem and Research Objectives...... 10

2.1Questions related research:...... 11 2.2Research Work Objectives:...... 12 Chapter3: Testing Background with Literature Review ...... 13

3.1Description of Testing:...... 13 3.2Testing for Software:...... 13 3.3Strategies to test the software:...... 14 3.4Approaches for testing Software:...... 15 3.5Application Software of Automatic Testing: ...... 16 3.6Web Testing Tools: ...... 17 3.7Description of Famous Testing Tools:...... 18 3.7.1Quick Testing Pro:...... 18 3.7.2Test Complete Testing Tool: ...... 19 3.7.3Selenium Tool for Testing:...... 19 3.8Integrated Development Environment of Selenium:...... 20 3.8.1Remote Control of Selenium:...... 20

vi 3.8.2Grid of Selenium: ...... 20 3.8.3Web Driver of Selenium:...... 21 3.9Features of Selenium Web Driver:...... 21 Chapter4: System Design and working Structure ...... 24

4.1Python :...... 24 4.2Python 3.7 version:...... 24 4.3PyCharm Framework: ...... 25 4.4Selenium Tool: ...... 25 4.5Mozilla :...... 25 4.6Gecko Driver:...... 25 4.7Test Cases of Selenium: ...... 26 4.7.1First Test Case: ...... 26 4.7.2Second Test Case:...... 26 4.7.3Third Test Case:...... 26 4.7.4Fourth Test Case:...... 26 4.7.5Fifth Test Case:...... 26 4.7.6Sixth Test Case:...... 26 4.8First Test Case Description: ...... 28 4.9Description of second Test Case:...... 29 4.10Third Test Case finding the title of web page:...... 30 4.11Fourth Test Case Description:...... 31 4.12Fifth Test Case: ...... 32 Chapter 5: Outcomes of Results and Analysis of the Results ...... 34

5.1The reporting for the results of Firefox Browser with selenium Web Driver:...... 34 5.1.1First Test case ...... 35 5.1.2Second Test Case:...... 36 5.1.3Third Test Case:...... 43 5.1.4Fourth Test Case:...... 47 5.1.5Fifth Test Case:...... 49 Conclusion ...... 51

Reference ...... 53

vii ABSTRACT

Testing is a vital factor for developing and managing the software. provides quality and insuring about the proper functioning of the software according to the defined scope of the software. Testing performed in two manners: manual and automatic .The manual testing having more disadvantages over automatic testing because it is related to the software experts, and a person can commit mistakes. So the testing based on manual basis is nearly not ical needs of testing dynamic software with high degree of testing. The manual testing was high cost testing and could even produces false results after second or third testing procedure on the same software. On the contrary automatic testing is best and efficient approach for fast and accurate testing in less time with less cost. Today the software required high level of testing because the users need high level of satisfaction and real time response. For these reasons automatic testing was performed, this testing was done by the software testing tool which works on automatic manner with the collaboration of any specified programming language. From theses automatic testing selenium was best option for testing the software (web browsers), because selenium having the capability to take the record of results and also having the friendly environment to test the cases. In this research thesis selenium is implemented on python programming language with its latest version Python 3.7 with the IDE of PyCharm. The python web driver manages the Mozilla Firefox browser, although Selenium provides the facility to run the test cases nearly on any browser and any but due to easiness and efficient results by default Mozilla Firefox browser was chosen. Finally after installing and setting all the defined tools test cases were prepared. In this research five core tests cases were designed and then implemented on the python enabled selenium web driver.

From these some of the test cases were also taken from the Microsoft Excel spreadsheet as a set of input values while some other test cases executed on individual values. The results and analysis shows that the selenium is an efficient automated testing tool specially when collaborated by the python programming language. And the overall results of each test Cases show that the automatic testing based on selenium and collaborated by the Python is an efficient, with respect of cost, time, and reliability, authentic and simple methodology.

viii CHAPTER 1 INTRODUCTION

Software testing is the important part of software development life cycle. It is related with the

Software testing phase is done before and after development of the software by the highly paid and expert software testers. It is estimated that the software testing is the most expensive and time consuming phase of development of the software. In most of the development organizations testing is done on manual basis by the experts and also the false testing results not only the time and resources but also the bad effects on the development organization. In that sense software testing plays an important role in the development and must be done in careful and timely manners. But most of the software development organizations perform the testing on the manual basis by the experts.(Wen, 2001). The manual testing has some deficits which takes time and mover cost over the individual project. Furthermore many types of applications exist which need to be tested on automatic basis in proper in this regard automatic testing has introduced in the world of information technology, which gives the software development approach a new meaning. In this chapter basic concepts on testing are defined with the types and finally the comparison between automatic and manual testing is done. [1]

1.1Software Testing:

Software testing is the crucial phase of software development life cycle as all the real implementation of software is based on the testing of the software. In the testing phase the quality of the software is tested according to the cost and level of satisfaction to the user with the services provided by the user. The testing is based on the cost and working environment of the software. For example high level of testing is done with critical and real time systems [5]

1.2Types of Testing:

Software are of different types and each of the software works in different environment for providing the specific services, so every software cannot be tested with same kind of testing and therefore different testing types are existed. Each type is suitable for software providing services. [3]

1 1.2.1Unit Testing:

The software is comprised of different working framework and developed by the different experts of related domain. Each of the developer related with specific domain develop the specific unit of the software and performs testing on his own. In case of correct testing unit of the software is ready to merge with the other developed units to work in the collective way. [2]

In this testing we just take small piece of testable software to check that the software is behaving like as we expect because it is more difficult for us to find errors in an integrated

1.2.2System/integration Testing:

Unit testing is the testing for specific part or unit of the software and cannot check the performance, reliability, throughput or efficiency of the software, while in the system testing all of the developed units of the software testing are merged into one software and such software is tested on behalf of the project manager, or customer by execution of the software but the testing of both stakeholders is of different nature(Burner et al., 2005)

There are three main categories of integration or system testing.

First is the big bang in this technique all the components of the system are integrated at once and then tested. Second is the bottom up in this technique the high level components are tested first, top down technique is the third one in which low level components are tested prior.

1.2.3White Box Testing:

White Box testing is known as the testing of technical experts who knows the structure of the software fully well and it is also be known as glass box testing because the internal working structure of the software is tested with the help of expert(S). [8] Both the developers and testers are involved in this testing because it helps them to understand that which of code is executed and which is not.

2 Figure 1 White box testing

1.2.4Black Box Testing:

The black box is done analyzing and testing the performance, working nature and proper customer of the software can perform such kind of testing. [8] In such testing the tester only knows about the input and the required output he is not concerned with the internal working of the system.

3 Figure 1.1 Black Box testing

Now this diagram needs some explanation as follows:

Step 1 Input: Requirement and functional specification of the systems are examined. High level design documents and application block source codes are also examined. Tester chooses valid inputs and rejects the invalid inputs.

Step 2 Processing Unit: Do not concerned with the internal working of the system. In processing unit tester constructs test cases with the selected input and execute them. If any defect is detected it will be fixed and re-tested.

Step 3 Output: After all these testing, tester gets desired output and prepares final report.

1.2.5Grey Box Testing:

This testing is based on both internal and external structures of the software, but to some extent not the like white or black box testing techniques which focus on the entire specific aspect of the software.

1.3Disadvantages of Manual Testing over Automated Testing:

Manual testing is the conventional testing method, which is time consuming and performed by the humans. This kind of testing cannot be performed in equal manners in all kind of situations with respect to time. Some of the Disadvantages of manual Testing are given below: Manual testing takes a lot of time and through this way the implementation of the

4 software is become late. This testing is only suitable for small test cases but due to the rapid application development requirements and usages of artificial intelligence features in the software, more test cases are required where this testing is not suitable. Manual testing is bases on human who have feelings and emotions and cannot perform the different testing with same enthusiasm level(Leitner et al., 2007).Manual testing is not surely correct in most ways which is not supporting the real time systems working may have work in the critical environment.

environment. Therefore automated tested has replaced the manual testing mechanism in many ways. It is suitable for many test cases with same level of accuracy and efficiency. It is based on the testing software which is not time consuming and also cost effective because it is estimated that the 80 % percent of the product cost is related with the software testing done on manual basis.

There are many software testing tools available like: Apium, Cucumber MIT testing tool, Selenium and silk test etc. Each of testing tool is suitable for different software for specific working environment of software. But selenium is best testing tool for web applications, as web applications are rapidly developed now a days(Au et al., 2008). In mostly referred as the web based applications and these applications are dynamic in nature with the requirements of efficient mechanism of testing. The web application should be built in rapid application scenario and no single testing can be applied on it. Also conventional testing or manual testing is not suitable for web applications with dynamic nature so in this situation automated tools for testing web applications are necessary Different web testing tools are shown in table below:

5 Web Latest cost Platform developer Reference testing version tools

Selenium 2.53.1 Open Web (Collaborati sourc based ve project) www.seleniumhq.org e

Eggplant Unknow £4,49 Web Test plant www.testplant.com/eggplant/test n 9 per based Ltd ing-tools/eggplant-developer/ Functional year

AutoIt 3.3.14.0 free windows Autoit www..com/automated- testing-tools.aspx

Hpwinrunn Unknow $999 Window Hewlett- www.telerik.com/automated- er n s Packard testing-tools.aspx

IcuTest 8.1 free WPF NXS-7 imacros.net Software Inc

iMacros 10.0.2 free windows iOpus

SilkTest 16.0 Window Micro Focus sourceforge.net/projects/sahi/ 0 s, Linux

Test Studio 2016 $2,9 Windo Telerik www.telerik.com/automated- testing-tools.aspx

Table 1.1 Advantage and Disadvantage of the automated testing

Selenium testing tool is best tool for web based applications; suitable description of selenium is given below:

6 1.4Selenium Automated Testing Tool For Web Based Application:

Selenium is an automated testing framework to test the web applications. It was developed by Jason Huggins in the year 2004 until 2012 it has same version from the developed year but in 2015 new version of selenium was introduced which is more flexible and easy to use with user friendly environment. The selenium testing tool is easy to use because a test case can be tested with same set of functionalities with different set of inputs(Jing and Yuqing, 2009). It is flexible tool because it can run on any operating system like Ubuntu, Linux, MAC or windows. It is basically written in programming language but it can be implemented on different programming language like python, Ruby, C# or Java itself. It is gives the XML file for test case and the comparison is done with the HTML file.

It gives the opportunity to developer to run the test cases in every famous web browsers like , Opera, fire fox or . It analyzes and tests the code and in cases of error occurrence it saves the information in spreadsheet where the actual code is also saved and the error prone code is pointed with the possible set of solutions. The process of execution is based on keyword as on the occurrence on keyword it starts the working (Wu, 2010)i The selenium is a package of tools and having set of components it component is used performing its own set of tasks. The components of selenium testing tool are described below:

1.5Selenium Integrated Development Environment:

Most of the software has an integrated development environment to interact with the usage of the services provided by specific software. The integrated environment for development of selenium is easy and user friendly as there is no need to be learnt the . The IDE of selenium provides the facilities to record, test, and even debug the test cases with efficient mechanism.[2][14]

1.5.1Selenium IDE:

In selenium codes from different languages like python, ruby, C# or java can be tested with the API calls and it is an easy way to implement the code of different language because it enhance the flexibility and working domain of the selenium. [5] It is implemented as Firefox add-on and allows the tester to record, playback and debugging the tests.

7 1.5.2Web Driver of Selenium:

Web driver of selenium collaborates with the running and exchange the set of version unlike the previous version which requires running the tests cases on java. [5]

1.5.3Selenium Remote Control:

Remote control of selenium testing tool provides the services to manage and execute the codes written in any programming languages but it supports some of the famous programming languages which are used to develop the web applications. It is supportive with the web driver and grid of selenium which saves the data into the HTML files for proper testing. [5]

1.5.4Selenium Grid:

It is a centralized server which combines the several servers from where the commands for different test cases are coming. Remote machines can access this server and test their codes on remote locations this gives the opportunity to run the different codes from different machines on the same unit of time. [5]

1.6Thesis Overview:

Software testing is definition of quality and working of the software. It analyzes, identifies is the definition of expected output with the given set of input. While developing any kind of software In any respective developing environment with specific programming language there are certain chances of error occurrence. The error are removed in the testing phase but due to the rapid application development of modern age requirement conventional or manual testing is less supported because it takes time and requires more cost. Furthermore web based applications are core need of modern world, such application are of dynamic nature which are less applicable for manual testing. So in that regard automated test tools are implemented. There is variety of test tools each of the tool is suitable for the specific type of software but selenium testing tool is much flexible and attractive testing tool for web based application. The web driver of the selenium gives the interoperability to run any code with the addition of required web browser. The selenium testing tool is supportive for both functionally and graphical user interface application. Different kinds of testing like white box or black box

8 CONCLUSION

Testing is important process of software devleopement life cycle which is done for the efficient development and maintaince of the software through out the life span of the software. Testiing is done foe insuring the proper fucntioning of software defined by scope of the software and providing plus defining the quality of the software. The higher testing is efficient , highers its quality. Software testing is done of two basis manners: manual and automatic. Manual testing having more disadvantages over automatic testin and cannot fulfills the needs of softaware with rapid testign and dynamic nature. Manual testing is a slow process and more probable of errors even in the repeative testing procedures. So in this thesis white box testing is produced on autmatic manner. The automatic testing is suitable for are in less time and reuced cost. It is done by the softtware testing tool. So in this work selenium a famous testing tool is used for performing the white box testing on the web pages which are of dynamic nature and need continuous testing . For implmenting selenium, its lates versin is installed which works in practical manner by enabling or adding python progamming language with it. Selenium implementation on python is much efficinet approach because python is advanced language which avoids the coding tiresome tasks which happen in the conventional programming.

For web browser selection Moxilla Firefox is used to working and automation of web browser, although selenium latest version supposts nearly all browsers but in this thesis but a little bit time taking browser. The whole codes are written in the PyCharm intergrated evironment which is the framework for python and used for creating and managing codes spcecially related to web pages. Finally in this thesis five test case are designed all the test cases are based on selenum testing on web pages. From the obtained results it can observed and analzed that the python enabaled selenim web driver is effcient in time and cost which improves the tesing process in automatic way. The test casee are based on simple validation or searchihg queries as Idea of basic qaueries can be further divided into much complicated queires of practical life. In some of the test case set of input values is taken either from the saved spreadsheet of microsoft Excel or indivually in the IDE of PyCharm. The test cases are further divided into the input values and each the source and results from each test input value of each test case given in pictorial format. The resutls and outcomes show that the selelnium is much efficient tool for testing on autoamtic basis and it also become more sophistaced when used wit the python becaseu python provides facility to reduced sized set of 51 codes. Automatic testing by python enabled web driver on webpages is efficient, realiable , efficinet in time, effcient in cost and accurate method for testing, validating and automating software(web browsers)

52