Software Engineering Lecture # 22 Quality Concepts

• Stages of Software Testing • Requirement Traceability Matrix Stages of Testing

 Unit testing/Component Testing/Module testing[2,3,6]

 Integration Testing

 System testing

 Acceptance testing Stages of Testing

• Unit Testing: Testing the individual components of the system. Involves the design of test cases that validate that the internal program logic of a given unit/component is functioning properly, and that program inputs produce valid outputs.

• Integration Testing: Concerned with testing the system as it is integrated from its components. Should focus on Interface testing where the interactions between sub-systems and components are tested. Stages of Testing • System Testing: Testing the system as a whole to validate that it meets its specification and the objectives of its users. The testing of a complete system prior to delivery. The purpose of system testing is to identify defects that will only surface when a complete system is assembled. System testing includes testing of performance, security, startup and recovery from failure modes.

• Acceptance Testing: User Acceptance Testing is a phase of software development in which the software is tested in the "real world" by the intended audience. Stages of Testing • Acceptance Testing includes: • Alpha Testing • Beta Testing

• Alpha testing – The alpha phase is the software prototype stage when the software is first able to run. It will not have all the intended functionality, but it will have core functions and will be able to accept inputs and generate outputs. An alpha test usually takes place in the developer's offices on a separate system.

• Beta Testing - The beta phase of software design exposes a new product, which has just emerged from in-house (alpha) testing, to a large number of real people, real hardware, and real usage. Beta testing is not a method of getting free software long-term, because the software expires shortly after the testing period. Requirement Traceability Matrix (RTM)

• A requirements traceability matrix is a document that traces and maps user requirements [requirement Ids from requirement specification document] with the test case ids.

• Purpose is to make sure that all the requirements are covered in test cases so that while testing no functionality can be missed.

• This document is prepared to make the clients satisfy that the coverage done is complete

• This document consists of Requirement/Base line doc Ref No., Test case/Condition, and Defects/Bug id. Using this document the person can track the Requirement based on the Defect id Types of RTM

• Forward Traceability – Mapping of Requirements to Test cases • Backward Traceability – Mapping of Test Cases to Requirements • Bi-Directional Traceability - A Good Traceability matrix is the references from test cases to basis documentation and vice versa. Types of RTM • Forward Traceability - This matrix is used to check whether the project progresses in the desired direction and for the right product. It makes sure that each requirement is applied to the product and that each requirement is tested thoroughly. It maps requirements to test cases.

• Backward Traceability - It is used to ensure whether the current product remains on the right track. The purpose behind this type of traceability is to verify that we are not expanding the scope of the project by adding code, design elements, test or other work that is not specified in the requirements. It maps test cases to requirements.

• Bi-Directional Traceability contains both Forward & Backward Traceability. This traceability matrix ensures that all requirements are covered by test cases and vice versa. Usefulness of RTM

• How can I certify that the final software product meets the customer’s needs?

• Now we can only make sure requirements are captured in the test cases by traceability matrix. Benefits of using RTM

• Make obvious to the client that the software is being developed as per the requirements. • To make sure that all requirements included in the test cases • To make sure that developers are not creating features that no one has requested • Easy to identify the missing functionalities. • If there is a change request for a requirement, then we can easily find out which test cases need to update. Disadvantages of NOT using RTM

• Poor or unknown test coverage, more defects found in production

• It will lead to miss some bugs in earlier test cycles which may arise in later test cycles. Then a lot of discussions arguments with other teams and managers before release.

• Difficult project planning and tracking, misunderstandings between different teams over project Steps to create RTM

• Make use of Microsoft Excel to create RTM

• Defining Columns (Requirement Id, Requirement description, Test Case Id)

• You can make RTM more effective by adding more columns RTM Format Creating RTM - Example

• Testers start writing their test scenarios/objectives and eventually the test cases based on some input documents – Business requirements document, Functional Specifications document and Technical design document (optional). Business Requirement Document (BRD) Functional Specification Document (FSD) • The below is the Functional Specifications document (FSD) based on the interpretation of the Business requirements document (BRD) and the adaptation of it to computer applications. Creating test cases for RTM

• Based upon the BRD and FSD, test scenarios are generated Creating RTM

• Such a document establishes a trace between, the BRD to FSD and eventually to the test scenarios. By creating a document like this, we can make sure every aspect of the initial requirements have been taken into consideration by the testing team for creating their test suites. Creating RTM

• You can leave it this way. However, in order to make it more readable, I prefer including the section names. This will enhance understanding when this document is shared with the client or any other teams. The outcome is as below: Creating RTM

• For each test scenario that you came up with, you are going to have at least 1 or more test cases. So, include another column when you get there and write the test case IDs as shows below: Creating RTM

• At this stage, the Traceability Matrix can be used to find gaps. For example, in the above Traceability Matrix you see that there are no test cases written for FSD section 1.2. • As a general rule, any empty spaces in the Traceability Matrix are potential areas for investigation. So a gap like this can mean one of the two things: • The test team has somehow missed considering the “Existing user” functionality. • The “Existing user” functionality has been deferred to later or removed from the application’s functionality requirements. In this case, the RTM shows an inconsistency in the FSD or BRD – which means that an update on FSD and/or BRD documents should be done. • If it is scenario 1, it will indicate the places where test team needs to work some more to ensure 100% coverage. • In scenarios 2, RTM not just shows gaps it points to incorrect documentation that needs immediate correction. • Let us now expand the RTM to include test case execution status and defects. Creating RTM

This version of the Traceability Matrix is generally prepared during or after test execution: Creating RTM

• The following are the important points to note about the Traceability Matrix: • 1) The execution status is also displayed. During execution, it gives a consolidated snapshot of how work is progressing. • 2) Defects: When this column is used to establish the backward traceability we can tell that the “New user” functionality is the most flawed. Instead of reporting that so and so test cases failed, RTM provides a transparency back to the business requirement that has most defects thus show casing the Quality in terms of what the client desires. • 3) As a further step, you can color code the defect ID to represent their states. For example, defect ID in red can mean it is still Open, in green can mean it is closed. When this is done, the RTM works as a health check report displaying the status of the defects corresponding to a certain BRD or FSD functionality is being open or closed. • 4) If there is a technical design document or use cases or any other artifacts that you would like to track you can always expand the above created document to suit your needs by adding additional columns. Example

• Let's understand the concept of Requirement Traceability Matrix through a banking project. On the basis of Business Requirement Document (BRD) and Technical Requirement Document (TRD), testers start writing test cases.

• Lets suppose, the following table is our Business Requirement Document or BRD for banking project. Here the scenario is that the customer should be able to login to xyz banking website with the correct password and user id while manager should be able to login to the website through customer login page. Business Requirement Document Technical Requirement Document

• The below table is our Technical Requirement Document

• Note: QA teams do not document the BRD and TRD. Also some companies use Function Requirement Documents (FRD) which are similar to Technical Requirement Document but the process of creating traceability matrix remains the same.

• Let's Go Ahead and create RTM Creating RTM

• Step 1: Our Test Case is • "Verify Login, when correct ID and Password is entered, it should login successfully" Creating RTM

• Step 2: Identify the Technical Requirement that this test case is verifying. For our test case, the technical requirement is T94 is being verified. Creating RTM

• Step 3: Note this Technical Requirement (T94) in the Test Case. Creating RTM

• Step 4: Identify the Business Requirement for which this TR (Technical Requirement-T94) is defined Creating RTM

• Step 5: Note the BR (Business Requirement) in Test Case Creating RTM

• Step 6: Do above for all Test Cases. Later Extract the First 3 Columns from your Test Suite. RTM is Ready! • Advantage of Requirement Traceability Matrix

• It confirms 100% test coverage • It highlights any missing requirements or document inconsistencies • It shows the overall defects or execution status with a focus on business requirements • It helps in analyzing or estimating the impact on the QA team's work with respect to revisiting or re-working on the test cases