Test Director 8.0 Process Handbook

Ver 1.5 TABLE OF CONTENTS

1.0 INTRODUCTION...... 3

2.0 ABOUT TESTDIRECTOR...... 3

3.0 MODULES IN TEST DIRECTOR...... 4

3.1 REQUIREMENTS...... 4 3.2 TEST PLAN...... 5 3.2.1 Customized Process...... 6 3.3 TEST LAB...... 9 3.3.1 Customized Process...... 10 3.4 DEFECTS...... 16 3.4.1 Customized Process...... 16 4. APPENDIX...... 17

5. CHANGE LOG...... 17 1.0 Introduction

The objective of this document is to –

 Serve as a quick-reference guide to know the standardized Test Director Process that we follow in 3MHIS Testing.  Serve as a Test Director Process Training material for 3MHIS Testing members  Standardize the use of Test Director across various projects in 3MHIS

The document also covers the various aspects like What is Test Director, Various modules in it, How the modules are customized to suit our needs, Who uses What, When to use what etc.

Note: This document is based on Test Director Version 8.0

2.0 About TestDirector

TestDirector is a Web-based Test Management Tool from Mercury Interactive Corporation. The Tool helps you organize and manage all phases of the application testing process, including specifying testing requirements, planning tests, executing tests, and tracking defects.

TestDirector guides you through the requirements specification, test planning, test execution, and defect tracking phases of the testing process. Test Director helps you create a framework and foundation for your testing workflow. TestDirector provides an intuitive and efficient method for scheduling and executing test sets, collecting test results, and analyzing the data.

TestDirector also features a sophisticated system for tracking application defects, enabling you to monitor defects closely from initial detection until resolution. TestDirector also offers integration with Mercury Interactive WinRunner, which would help us run automated WinRunner script from Test director itself and analyze results. 3.0 Modules in Test Director

3.1 Requirements

Requirements describe in detail what needs to be tested in your application and provide the test team with the foundation on which the entire testing process is based. Requirements module in Test Director help us create, define and analyze the requirements. The Requirements are then linked to tests and defects to provide complete traceability and aid the decision-making process.

As part of our current Process, we are NOT planning to utilize this module. However, the ultimate goal is to get all the requirements here (either import from word or give a connection from Doors) and ensure a complete end to end traceability is maintained).

Efforts are underway to pilot this module usage on CGS project. We have sample requirements imported into Test Director already for CGS. 3.2 Test Plan

Test Plan module in Test Director help us define a hierarchical framework for maintaining the Test Cases.Test Cases are organized hierarchically with required parent folders forming a Test plan tree (graphical representation of your test case hierarchy). Test cases are built by defining test steps: detailed, step by-step instructions on how to execute a test. A step includes the actions to perform, the input to enter, and the expected output. 3.2.1 Customized Process

Test Plan module has been customized to suit 3MHIS Testing process. The current process that we follow is outlined below.

3.2.1.1 Test Plan Tree

Folders will be organized based on the high-level category and NOT based on the product release version. For ex: we’ll have folders ‘Product Specific’, ‘Components’ at high-level and required sub-folders under ‘product specific’ like ‘Batch’, ‘Interactive’ will be created for better identification of test cases.

Note: For better reporting purposes, we’ll limit the depth of folder structure we create in Test Plan area (The no of folders should be as low as possible).

3.2.1.2 Test Details Tab

As part of our current process, we’ll maintain the following System and User Defined fields in the Details tab –

 Test Name: This is a System Defined field and it is populated automatically when a test is created.

 Creation Date: This is a System Defined field and it is populated automatically when a test is created.

 Designer: This is a System Defined field of type ‘User list’. It is the responsibility of the Test Designer to select the correct value while test case creation

 Status: This is a System Defined field of type ‘Look up list’ and is primarily used as part of the Automation Maintenance Process. The Default system generated values has to be removed and the new set of values listed below to be maintained –

 Design – The Designer will set this status when the test case is still under construction and not ready for testing.  New - The Designer will set this status when a New test case has been added and is ready for testing. The status will be changed to ‘Baselined’ when Offshore Team completes manual execution of this test case or automating it.  Modified - The Designer will set this status when an existing test case has been modified. The status will be changed to ‘Baselined’ when Offshore Team completes manual execution of the modified test case or completes modifying the automated Test script associated with this test case.  Baselined – This is final status that any test case attains. By default all existing cases will be in ‘Baselined’ status.

Note: More detailed information on how this field is being used is explained in the Automation Maintenance process (see Appendix section)

 Coverage: This is a User Defined field of type List Box. The idea of using this field is to define what the test case level of coverage is. This is a mandatory field and it holds one of the values given below:

 1 - Primary: Any Test case that falls under the base set that has to be must- covered takes this value.  2 - Secondary: Any Test case that is considered to be an extended coverage type takes this value.  3 - Optional: Any time permits/non-critical kind of test case takes this value.

 Test Category: This is a User Defined field of type List Box. The idea of using this field is to easily identify the category in which a test case falls into (Editing, Grouping, etc.). This is a mandatory field and it holds one of the 37 values attached below.

"Test Category.xls"

Note: Any New value addition to be duly communicated and changes to be made for all products.

 Platform: This is a User Defined filed of type List Box. The idea of using this field is to identify platform(s) a test case is applicable for.

 ALL – Test case applicable for all supported platforms  AS400 only – Test case applicable for AS400 only  PC only – Test case applicable for PC only  MF only – Test case applicable for MF only  Sun only – Test case applicable for Sun only  HP only – Test case applicable for HP only  AS400, MF – Test case applicable for AS400, MF only  AS400, PC - Test case applicable for AS400, PC only  PC, AS400 - Test case applicable for PC, AS400 only  PC, MF – Test case applicable for PC, MF only  PC, Sun - Test case applicable for PC, Sun only  PC, HP - Test case applicable for PC, HP only  MF, SUN - Test case applicable for MF, Sun only  MF, HP - Test case applicable for MF, HP only  SUN, HP - Test case applicable for SUN, HP only  Remedy Ref: This is a User Defined field of type free text, which will only allow 6- numeric characters. The idea of using this field is to associate any associated Remedy Defect Reference number to the test case.

 Reviewed by: This is a field of type ‘User List’ and needs to be filled in whenever any review of test case happens. This field should hold the Reviewer’s Name.

 Reviewed on: This is a field of type ‘Date’ and needs to be filled in whenever any review of test case happens. This field should hold the test case Review Date.

A Snapshot of the customized Test Plan section is show below: 3.3 Test Lab

After you design tests in the Test Plan module, you create a test sets tree in Test Lab module. A test sets tree enables you to organize your testing process by grouping test sets in folders and organizing them in different hierarchical levels. Test Lab module is where you create Test Sets to organize test execution and perform Test Runs. A test set contains a set of tests that is logically grouped to achieve specific testing goals. Once you have defined test sets, you can execute the tests both Manually or using automated tools and update test results.

3.3.1 Customized Process

Test Lab module has been customized to suit 3MHIS Testing process. The current process that we follow is outlined below.

3.3.1.1 Test Sets Tree

One folder per release to be created based on the release name. For ex: we’ll have a folder per release ‘April Release 2004’, Only limited number of Test sets will be created. For ex: We’ll have one test set for ‘Components’ and one for ‘Product Specific’.

Note: No segregation of Test set by Platform will be made.

3.3.1.2 Test Execution Grid Tab

As part of our current process, we’ll maintain the following System and User Defined fields in the Execution Grid and use it as applicable for products.

 Plan: Test Name: This is a System defined field that holds the name of the test case and it is automatically referenced from the Test Plan module.

 Plan: Type: This is a System defined field displays the type of test case ‘Manual’ or ‘Automated’ and it is automatically referenced from the Test Plan module.

 Exec Priority: This is a User Defined field of type list box. The idea of using this field is to identify those cases, which are of higher priority to be executed. This field will hold one of the values  1 – High Priority  2 - Medium Priority  3 – Low Priority

Note: It is the 3M QA Responsibility to prioritize.

 Build#: This is a User Defined field of type free text. The idea of using this field is to identify which build the test case is to be run on.

Note: It is the responsibility of the Onsite QA Team to enter the Build No.  PC Status: This is a User Defined field of type list box. The idea of using this field is to mark the PC platform specific testing Status. This field will have one of the following values:  Failed – Test case failed  N/A – Not Applicable  No Run – Yet to be Run  Not Completed – Test Started, but yet to be completed  Passed – Test case Passed

Note: It is the responsibility of the tester “Offshore/Onsite” who executes the test case to manually enter the appropriate value. However, if a test case is Not Applicable for the platform the value ‘N/A’ will be set by the 3M Onsite QA Team.

 PC Tester: This is a User Defined field of type User list. It is the responsibility of the tester “Offshore/Onsite” who executes the test case to manually enter the Name of the Tester who executed the Test case.

 PC Exec Date: This is a User Defined field of type Date. It is the responsibility of the tester “Offshore/Onsite” who executes the test case to manually enter the date on which the test case was executed.

 AS400 Status: This is a User Defined field of type list box. The idea of using this field is to mark the AS400 platform specific testing Status. This field will have one of the following values:  Failed – Test case failed  N/A – Not Applicable  No Run – Yet to be Run  Not Completed – Test Started, but yet to be completed  Passed – Test case Passed

 AS400 Tester: This is a User Defined field of type User list. It is the responsibility of the tester “Offshore/Onsite” who executes the test case to manually enter the Name of the Tester who executed the Test case.

 AS400 Exec Date: This is a User Defined field of type Date. It is the responsibility of the tester “Offshore/Onsite” who executes the test case to manually enter the date on which the test case was executed.

 MF Status: This is a User Defined field of type list box. The idea of using this field is to mark the Mainframe platform specific testing Status. This field will have one of the following values:  Failed – Test case failed  N/A – Not Applicable  No Run – Yet to be Run  Not Completed – Test Started, but yet to be completed  Passed – Test case Passed

Note: It is the responsibility of the tester “Offshore/Onsite” who executes the test case to manually enter the appropriate value. However, if a test case is Not Applicable for the platform the value ‘N/A’ will be set by the 3M Onsite QA Team.  MF Tester: This is a User Defined field of type User list. It is the responsibility of the tester “Offshore/Onsite” who executes the test case to manually enter the Name of the Tester who executed the Test case.

 MF Exec Date: This is a User Defined field of type Date. It is the responsibility of the tester “Offshore/Onsite” who executes the test case to manually enter the date on which the test case was executed.

 HP Status: This is a User Defined field of type list box. The idea of using this field is to mark the HP platform specific testing Status. This field will have one of the following values:  Failed – Test case failed  N/A – Not Applicable  No Run – Yet to be Run  Not Completed – Test Started, but yet to be completed  Passed – Test case Passed

 HP Tester: This is a User Defined field of type User list. It is the responsibility of the tester “Offshore/Onsite” who executes the test case to manually enter the Name of the Tester who executed the Test case.

 HP Exec Date: This is a User Defined field of type Date. It is the responsibility of the tester “Offshore/Onsite” who executes the test case to manually enter the date on which the test case was executed.

 SUN Status: This is a User Defined field of type list box. The idea of using this field is to mark the Sun platform specific testing Status. This field will have one of the following values:  Failed – Test case failed  N/A – Not Applicable  No Run – Yet to be Run  Not Completed – Test Started, but yet to be completed  Passed – Test case Passed

 SUN Tester: This is a User Defined field of type User list. It is the responsibility of the tester “Offshore/Onsite” who executes the test case to manually enter the Name of the Tester who executed the Test case.

 SUN Exec Date: This is a User Defined field of type Date. It is the responsibility of the tester “Offshore/Onsite” who executes the test case to manually enter the date on which the test case was executed.  Client OS: This is a User Defined field of type List Box This field will be primarily used to determine any specific client OS is to be used. Either Onsite 3M QA/ Offshore Team will fill in this field. This field will have one of the following values

 N/A: All the Test cases will default to N/A. This status will be set to those cases where platform doesn’t matter or not applicable.  Win 98: Client OS is Windows 98.  Win XP: Client OS is Windows XP.  Win 2000: Client OS is Windows 2000.

 Network OS: This is a User Defined field of type List Box This field will be primarily used to determine which specific Server OS is to be used. Either Onsite 3M QA/ Offshore Team will fill in this field. This field will have one of the following values

 N/A: All the Test cases will default to N/A. This status will be set to those cases where platform doesn’t matter or not applicable.  Citrix: Server platform is Citrix  Novell: Server Platform is Netware  NT 4.0 Server: Server OS is NT 4.0  Terminal Server: Server Platform is Terminal Server  Win 2000: Windows 2000 is used as Server  Windows 2000 Server: Server OS is Windows 2000 Server  Windows 2003 Server (Data C. Ed): Server OS is Windows 2003 Data Center Edition Server  Windows 2003 Server (Entrpr. Ed): Server OS is Windows 2003 Enterprise Edition Server  Windows 2003 Server (Std. Ed): Server OS is Windows 2003 Standard Edition Server  Win NT: Windows NT is used as a Server  Win XP: Windows XP is used as a Server

 Remedy Number: This is a User Defined field of type free text. The idea of using this field is to enter any associated Remedy Defect Number.

 TD Defect Number: This is a User Defined field of type free text. The idea of using this field is to enter any associated TD Defect Number.

 Compare Method: This is a User Defined field of type list-box. The idea of using this field is to determine which compare method to use. The option to select from will be 1) Build-to-Build 2) Release-to-Release 3) CER and 4) PID Compare.

 Temp Status: This is a System Defined field by original name ‘Status’ renamed to ‘Temp Status’. This is a Temporary field and it is has to be ignored.

Note: The idea of using this field is to hold any temporary status value of a test run. We are using a custom PC Status column because each time a tester Runs a test case, it would update the Status column. So say PC passed, they run the test case again on AS400 and it failed, the Status column would automatically reflect the last run. To avoid this, a custom PC Status field has been created that will not be automatically tied to each Run.

 Temp Tester: This is a System Defined field by original name ‘Tester’ renamed to ‘Temp Tester’. This is a Temporary field and it is has to be ignored.  Temp Exec Date: This is a System Defined field by original name ‘Exec Date’ renamed to ‘Temp Exec Date’. This is a Temporary field and it is has to be ignored.

Note: There are also other fields, which we don’t use, but still present in Test Director. Ignore those fields and set your view to only those fields that we use as part of our process. Sometimes, all the fields that are part of our process may not be applicable for all products. For ex: GPS doesn’t have any other platform other than windows. So, fields like Mf Status, MF Tester, MF Exec Date etc., can be hidden from the view. Favorite views will be set for each the product that would present the applicable fields for the product.

A Snapshot of the customized Test Lab section is show below: 3.3.1.3 Run Window

This is the window that pops when you click on ‘Run’ from the Test Execution grid.

As part of our current process, we’ll maintain the following System and User Defined fields -.

 Run Name: This is a System Defined filed of type free text. The idea of using this field to uniquely identify the Test Run. Test Director automatically populates this filed with a unique name. Accept defaults or you can manually enter any name to identify the test run.

 Platform: This is a User Defined Field of type List Box The Idea of using this filed is to segregate the test run by platform to compare the results of all runs.. Each time Tester Runs a test case, and say a test step fails in PC and he updates the Actual Result column. Next Time he runs the same test case in AS400 and it passes. In this case, when you have to compare results of all runs, this platform field would be handy.

It is the responsibility of the tester “Offshore/Onsite” who executes the test case to manually enter the appropriate value. This field will have one of the following values

 PC – Windows Platform  MF – Mainframe Platform  AS400 – As400 Platform  HP – HP Platform  Sun – Sun Platform

A Snapshot of the customized Test Run window is show below: 3.3.1.4 Associating Tests to Defect

Tests needs to be associated with defects to ensure traceability. When you execute any test case and detect a defect, you can associate the test run to the defect by clicking on the ‘Add Defect’ button in the Test Run window. Once you add a defect via the Test Run window TestDirector automatically creates an association between the test run and the new defect. This will help us identify what test case needs to be run to test any defect.

Note: It is the responsibility of the tester “Offshore/Onsite” who executes the test case and enters a defect to associate the test with the Defect. Refer Test Director user guide for more details.

3.4 Defects

You use the Test Director Defects module to Add Defects and then track the defects end to end until closure.

3.4.1 Customized Process

The Defects module has also been customized to suit 3MHIS Testing process. Refer to the attached Defect Methodology document for the process that we follow. 4. Appendix

4.1 Automation Maintenance - Test Director Process

The attached document explains what process needs to be followed from a Test Director perspective to facilitate effective maintenance of Automated scripts

"Automation Maintenance TD Process.doc"

4.2 Automation Maintenance – Script Process

The attached document explains what process needs to be followed from a script maintenance perspective to facilitate effective maintenance of Automated scripts

"Script Maintenance Process.doc"