Scenario Testing
Prepared By S.Rijutha AP/IT
1 IT6004/ST/Unit-3 What is it? set of realistic user activities that are used for evaluating the product. Testing involves customer scenarios Methods
System Scenarios Use case Scenarios
Step wise procedure on Testing covers several how a user intends to use components in the system the system
2 IT6004/ST/Unit-3 System Scenarios
Story line
Life cycle/state transitions
Deployment/implementation stories from customer
Business verticals
Battle ground
3 IT6004/ST/Unit-3 Story line Develop a story line that combines various activities of the product that may be executed by an end user. A user enters his or her office, logs into the system, checks mail, responds to some mails, compiles some programs, performs unit testing and so on.
4 IT6004/ST/Unit-3 Life cycle/state transitions Consider an object, derive the different transitions/modifications that happen to the object, and derive scenarios to cover them. For example, in a savings bank account, you can start with opening an account with a certain amount of money, make a deposit, perform a withdrawal, and calculate interest, and so on.
5 IT6004/ST/Unit-3 Deployment/implementation stories from customer Develop a scenario from a known customer deployment/implementation details and create a set of activities by various users in that implementation.
6 IT6004/ST/Unit-3 Business verticals Visualize how a product/software will be applied to different verticals and create a set of activities as scenarios to address specific vertical businesses. For example, take the purchasing function. It may be done differently in different verticals like pharmaceuticals, software houses, and government organizations. Visualizing these different types of tests make the product "multi- purpose."
7 IT6004/ST/Unit-3 Battle ground Create some scenarios to justify that "the product works" and some scenarios to "try and break the system" to justify "the product doesn't work."
8 IT6004/ST/Unit-3 The set of scenarios developed will be more effective if the majority of the approaches mentioned above are used in combination, not in isolation. Scenario should not be a set of disjointed activities which have no relation to each other. Any activity in a scenario is always a continuation of the previous activity, and depends on or is impacted by the results of previous activities. A right mix of scenarios using the various approaches explained above is very critical for the effectiveness of scenario testing.
9 IT6004/ST/Unit-3 Coverage of activities by scenario testing
10 IT6004/ST/Unit-3 Use Case Scenario A use case scenario is a stepwise procedure on how a user intends to use a system, with different user roles and associated parameters. A use case scenario can include stories, pictures, and deployment details.
Actors • Users with different roles
• What the product should do for a particular system behavior activity
• Users with a specific role to interact between the Agents actors and the system
11 IT6004/ST/Unit-3 Example needs to be tested not concerned about before applying the the logic of how the sequence of agent computer works Agent activities and actor activities.
Query Cheque
System Actor Response
Need not know what the Response official is doing and what command he is using to Cash interact with the computer.
12 IT6004/ST/Unit-3 Actor and System response in use case for ATM cash withdrawal Use cases are useful for explaining customer problems and how the software can solve those problems without any ambiguity. Actor System response Users likes to withdraw cash and inserts the Request password or PIN card in the ATM machine User fills in the password or PIN Validate the password or PIN Give a list containing types of accounts User selects an account type Ask the user for amount to withdraw User fills in the amount of cash required Check availability of funds Update account balance Prepare receipt Dispense cash Retrieve cash from ATM Print receipt
13 IT6004/ST/Unit-3 This way of documenting a scenario and testing makes it simple and also makes it realistic for customer usage. Use cases are not used only for testing. In some product implementations, use cases are prepared prior to the design and coding phases, and they are used as a set of requirements for design and coding phases. All development activities are performed based on use case documentation. In extreme programming models these are termed as user stories and form the basis for architecture/design and coding phases. Hence, use cases are useful in combining the business perspectives and implementation detail and testing them together.
14 IT6004/ST/Unit-3 Defect Bash
15 IT6004/ST/Unit-3 Defect Bash Defect bash is an ad hoc testing where people performing different roles in an organization test the product together at the same time. This is very popular among application development companies, where the product can be used by people who perform different roles What is to be tested is left to an individual's decision and creativity
16 IT6004/ST/Unit-3 Brings in the Good practices Cross boundaries and test beyond assigned areas Testing isn't for testers alone Letting everyone in the organization use the product before delivery— "Eat your own dog food" Fresh eyes have less bias Users of software are not same Does testing wait till all documentation is done? Testing isn't to conclude the system works or doesn't work
17 IT6004/ST/Unit-3 Steps involved
Step 1 • Choosing the frequency and duration of defect bash
Step 2 • Selecting the right product build
Step 3 • Communicating the objective of each defect bash to everyone
Step 4 • Setting up and monitoring the lab for defect bash
Step 5 • Taking actions and fixing issues
Step 6 • Optimizing the effort involved in defect bash
18 IT6004/ST/Unit-3 Step 1 • Choosing the frequency and duration of defect bash
Defect bash is an activity involving a large amount of effort and an activity involving huge planning Frequent defect bashes low return on investment, and too few defect bashes may not meet the objective of finding all defects. Duration is also an important factor. Optimizing the small duration is a big saving as a large number of people are involved. On the other hand if the duration is small, the amount of testing that is done may not meet the objective.
19 IT6004/ST/Unit-3 Step 2 • Selecting the right product build
A good quality build is needed for defect bash. gives confidence on the product and progress A regression tested build would be ideal as all new features and defect fixes would have been already tested in such a build. An intermediate build where the code functionality is evolving or an untested build will make the purpose and outcome of a defect bash ineffective. Also, when testers doing a defect bash uncover an excessive number of defects or very severe defects, the confidence of the tester’s falls and the perception of the product being unstable linger on for long.
20 IT6004/ST/Unit-3 Step 3 • Communicating the objective of each defect bash to everyone
The objective should be to find a large number of uncovered defects or finding out system requirements (CPU, memory, disk, and so on) or finding the non-reproducible or random defects, which could be difficult to find through other planned tests. Once they are told in advance, the members of defect bash team will be in a better position to contribute towards stated objectives.
21 IT6004/ST/Unit-3 Step 4 • Setting up and monitoring the lab for defect bash
Finding out the right configuration, resources (hardware, software, and set of people to perform defect bash) are activities that have to be planned carefully before a bash actually starts.
Right Set up is done- most critical to ensure
Else test fails
22 IT6004/ST/Unit-3 During defect bash, the product parameters and system resources (CPU, RAM, disk, network) need to be monitored for defects and also corrected so that users can continue to use the system for the complete duration of the defect bash. However, if the lab is not set up properly or not monitored properly, there is a chance that some of the non-functional defects may not get noticed at all.
memory leak, long Types of defects turnaround time, that are in the missed requests, product, as high impact and utilization of system reported by the Non Functional Functional Defect resources users Defect
23 IT6004/ST/Unit-3 Step 5 • Taking actions and fixing issues
different interpretations of the same defect by different users It is difficult to solve all the problems if they are taken one by one and fixed in code Classification of defect need to be done. Defect associated with issue. All the issues reported from a defect bash need to be taken through a complete code and design inspections, analyzed, and fixed together in places where a defect could evolve from.
24 IT6004/ST/Unit-3 Step 6 • Optimizing the effort involved in defect bash
Keeping the right setup, sharing the objectives, and so on, to save effort and meet the purpose have been already discussed. Conduct micro lever defect bashes before conducting one on a large scale.
Defect Bash
Feature/component Integration defect Product defect bash defect bash bash
25 IT6004/ST/Unit-3 Three product defect bashes conducted in two hours with 100 people Total effort involved is 3*2*100= 600 person hours. If the feature/component test team and integration test team, that has 10 people each, can participate in doing two rounds of micro level bashes, which can find out one third of defects that are expected, then effort saving is 20% with respect to the following calculation. Total effort involved in two rounds of product bashes -400 man hours Effort involved in two rounds of feature bash (2*2*10)-40 Effort involved in two rounds of integration bash (2*2*10)-40 Effort saved = 600 - (A+ B + C) = 600 - 480 =120 person hours, or 20%
26 IT6004/ST/Unit-3 System Testing
27 IT6004/ST/Unit-3 Goal and types The goal is to ensure that Functional testing the system performs Performance testing according to its requirements. Stress testing System test evaluates both functional behavior and Configuration testing quality requirements such Security testing as reliability, usability, performance and security. Recovery testing
28 IT6004/ST/Unit-3 A load is a series of inputs that simulates a group of transactions. A transaction is a unit of work seen from the system user’s view . A transaction consists of a set of operations that may be performed by a person, software system, or a device that is outside the system.
29 IT6004/ST/Unit-3 Functional testing
Ensure that the behavior of the system adheres to the requirements specification. Goals All types or classes of legal inputs must be accepted by the software. All classes of illegal inputs must be rejected (however, the system should remain available). All possible classes of system output must exercise and examined. All effective system states and state transitions must be exercised and examined. All functions must be exercised.
30 IT6004/ST/Unit-3 Performance testing
Functional requirements. Users describe what functions the software should perform. Quality requirements. There are nonfunctional in nature but describe quality levels expected for the software. One example of a quality requirement is performance level. The goal of system performance tests is to see if the software meets the performance requirements. Performance testing allows testers to tune the system; that is, to optimize the allocation of system resources. Resources for performance testing must be allocated in the system test plan
31 IT6004/ST/Unit-3 Performance objectives must be articulated clearly by the users/clients in the requirements documents, and be stated clearly in the system test plan. The objectives must be quantified. Results of performance tests are quantifiable. Resources A source of transactions to drive the experiments. An experimental testbed that includes hardware and software the system-under-test interacts with. Instrumentation or probes that help to collect the performance data. A set of tools to collect, store, process, and interpret the data.
32 IT6004/ST/Unit-3 33 IT6004/ST/Unit-3 Stress testing
When a system is tested with a load that causes it to allocate its resources in maximum amounts, this is called stress testing. For example, if an operating system is required to handle 10 interrupts/second and the load causes 20 interrupts/second, the system is being stressed. The goal of stress test is to try to break the system; find the circumstances under which it will crash. This is sometimes called “breaking the system.”
34 IT6004/ST/Unit-3 Configuration testing
Configuration testing allows developers/testers to evaluate system performance and availability when hardware exchanges and reconfigurations occur Objectives: Show that all the configuration changing commands and menus work properly. Show that all interchangeable devices are really interchangeable, and that they each enter the proper states for the specified conditions. Show that the systems’ performance level is maintained when devices are interchanged, or when they fail.
35 IT6004/ST/Unit-3 Types of operations Rotate and permutate the positions of devices to ensure physical/ logical device permutations work for each device Induce malfunctions in each device, to see if the system properly handles the malfunction; Induce multiple device malfunctions to see how the system reacts.
36 IT6004/ST/Unit-3 Security testing
Designing and testing software systems to insure that they are safe and secure.
37 IT6004/ST/Unit-3 Effects of security breaches loss of information; corruption of information; misinformation; privacy violations; denial of service.
38 IT6004/ST/Unit-3 Areas to focus on during security testing Password Checking Legal and Illegal Entry with Passwords Password Expiration Encryption Browsing Trap Doors Viruses
39 IT6004/ST/Unit-3 Recovery testing
Recovery testing subjects a system to losses of resources in order to determine if it can recover properly from these losses. Example: on-line banking software. Areas to focus on Restart Switch over All transactions and processes must be carefully examined to detect: loss of transactions; merging of transactions; incorrect transactions; an unnecessary duplication of a transaction.
40 IT6004/ST/Unit-3