Scenario Testing Prepared By S.Rijutha AP/IT 1 IT6004/ST/Unit-3 What is it? set of realistic user activities that are used for evaluating the product. Testing involves customer scenarios Methods System Scenarios Use case Scenarios Step wise procedure on Testing covers several how a user intends to use components in the system the system 2 IT6004/ST/Unit-3 System Scenarios Story line Life cycle/state transitions Deployment/implementation stories from customer Business verticals Battle ground 3 IT6004/ST/Unit-3 Story line Develop a story line that combines various activities of the product that may be executed by an end user. A user enters his or her office, logs into the system, checks mail, responds to some mails, compiles some programs, performs unit testing and so on. 4 IT6004/ST/Unit-3 Life cycle/state transitions Consider an object, derive the different transitions/modifications that happen to the object, and derive scenarios to cover them. For example, in a savings bank account, you can start with opening an account with a certain amount of money, make a deposit, perform a withdrawal, and calculate interest, and so on. 5 IT6004/ST/Unit-3 Deployment/implementation stories from customer Develop a scenario from a known customer deployment/implementation details and create a set of activities by various users in that implementation. 6 IT6004/ST/Unit-3 Business verticals Visualize how a product/software will be applied to different verticals and create a set of activities as scenarios to address specific vertical businesses. For example, take the purchasing function. It may be done differently in different verticals like pharmaceuticals, software houses, and government organizations. Visualizing these different types of tests make the product "multi- purpose." 7 IT6004/ST/Unit-3 Battle ground Create some scenarios to justify that "the product works" and some scenarios to "try and break the system" to justify "the product doesn't work." 8 IT6004/ST/Unit-3 The set of scenarios developed will be more effective if the majority of the approaches mentioned above are used in combination, not in isolation. Scenario should not be a set of disjointed activities which have no relation to each other. Any activity in a scenario is always a continuation of the previous activity, and depends on or is impacted by the results of previous activities. A right mix of scenarios using the various approaches explained above is very critical for the effectiveness of scenario testing. 9 IT6004/ST/Unit-3 Coverage of activities by scenario testing 10 IT6004/ST/Unit-3 Use Case Scenario A use case scenario is a stepwise procedure on how a user intends to use a system, with different user roles and associated parameters. A use case scenario can include stories, pictures, and deployment details. Actors • Users with different roles • What the product should do for a particular system behavior activity • Users with a specific role to interact between the Agents actors and the system 11 IT6004/ST/Unit-3 Example needs to be tested not concerned about before applying the the logic of how the sequence of agent computer works Agent activities and actor activities. Query Cheque System Actor Response Need not know what the Response official is doing and what command he is using to Cash interact with the computer. 12 IT6004/ST/Unit-3 Actor and System response in use case for ATM cash withdrawal Use cases are useful for explaining customer problems and how the software can solve those problems without any ambiguity. Actor System response Users likes to withdraw cash and inserts the Request password or PIN card in the ATM machine User fills in the password or PIN Validate the password or PIN Give a list containing types of accounts User selects an account type Ask the user for amount to withdraw User fills in the amount of cash required Check availability of funds Update account balance Prepare receipt Dispense cash Retrieve cash from ATM Print receipt 13 IT6004/ST/Unit-3 This way of documenting a scenario and testing makes it simple and also makes it realistic for customer usage. Use cases are not used only for testing. In some product implementations, use cases are prepared prior to the design and coding phases, and they are used as a set of requirements for design and coding phases. All development activities are performed based on use case documentation. In extreme programming models these are termed as user stories and form the basis for architecture/design and coding phases. Hence, use cases are useful in combining the business perspectives and implementation detail and testing them together. 14 IT6004/ST/Unit-3 Defect Bash 15 IT6004/ST/Unit-3 Defect Bash Defect bash is an ad hoc testing where people performing different roles in an organization test the product together at the same time. This is very popular among application development companies, where the product can be used by people who perform different roles What is to be tested is left to an individual's decision and creativity 16 IT6004/ST/Unit-3 Brings in the Good practices Cross boundaries and test beyond assigned areas Testing isn't for testers alone Letting everyone in the organization use the product before delivery— "Eat your own dog food" Fresh eyes have less bias Users of software are not same Does testing wait till all documentation is done? Testing isn't to conclude the system works or doesn't work 17 IT6004/ST/Unit-3 Steps involved Step 1 • Choosing the frequency and duration of defect bash Step 2 • Selecting the right product build Step 3 • Communicating the objective of each defect bash to everyone Step 4 • Setting up and monitoring the lab for defect bash Step 5 • Taking actions and fixing issues Step 6 • Optimizing the effort involved in defect bash 18 IT6004/ST/Unit-3 Step 1 • Choosing the frequency and duration of defect bash Defect bash is an activity involving a large amount of effort and an activity involving huge planning Frequent defect bashes low return on investment, and too few defect bashes may not meet the objective of finding all defects. Duration is also an important factor. Optimizing the small duration is a big saving as a large number of people are involved. On the other hand if the duration is small, the amount of testing that is done may not meet the objective. 19 IT6004/ST/Unit-3 Step 2 • Selecting the right product build A good quality build is needed for defect bash. gives confidence on the product and progress A regression tested build would be ideal as all new features and defect fixes would have been already tested in such a build. An intermediate build where the code functionality is evolving or an untested build will make the purpose and outcome of a defect bash ineffective. Also, when testers doing a defect bash uncover an excessive number of defects or very severe defects, the confidence of the tester’s falls and the perception of the product being unstable linger on for long. 20 IT6004/ST/Unit-3 Step 3 • Communicating the objective of each defect bash to everyone The objective should be to find a large number of uncovered defects or finding out system requirements (CPU, memory, disk, and so on) or finding the non-reproducible or random defects, which could be difficult to find through other planned tests. Once they are told in advance, the members of defect bash team will be in a better position to contribute towards stated objectives. 21 IT6004/ST/Unit-3 Step 4 • Setting up and monitoring the lab for defect bash Finding out the right configuration, resources (hardware, software, and set of people to perform defect bash) are activities that have to be planned carefully before a bash actually starts. Right Set up is done- most critical to ensure Else test fails 22 IT6004/ST/Unit-3 During defect bash, the product parameters and system resources (CPU, RAM, disk, network) need to be monitored for defects and also corrected so that users can continue to use the system for the complete duration of the defect bash. However, if the lab is not set up properly or not monitored properly, there is a chance that some of the non-functional defects may not get noticed at all. memory leak, long Types of defects turnaround time, that are in the missed requests, product, as high impact and utilization of system reported by the Non Functional Functional Defect resources users Defect 23 IT6004/ST/Unit-3 Step 5 • Taking actions and fixing issues different interpretations of the same defect by different users It is difficult to solve all the problems if they are taken one by one and fixed in code Classification of defect need to be done. Defect associated with issue. All the issues reported from a defect bash need to be taken through a complete code and design inspections, analyzed, and fixed together in places where a defect could evolve from. 24 IT6004/ST/Unit-3 Step 6 • Optimizing the effort involved in defect bash Keeping the right setup, sharing the objectives, and so on, to save effort and meet the purpose have been already discussed. Conduct micro lever defect bashes before conducting one on a large scale. Defect Bash Feature/component Integration defect Product defect bash defect bash bash 25 IT6004/ST/Unit-3 Three product defect bashes conducted in two hours with 100 people Total effort involved is 3*2*100= 600 person hours. If the feature/component test team and integration test team, that has 10 people each, can participate in doing two rounds of micro level bashes, which can find out one third of defects that are expected, then effort saving is 20% with respect to the following calculation.