A Taxonomy for Scenario Use in Requirements Elicitation and Analysis of Software Systems

Brian D. Chance Bonnie E. Melhart Motorola Cellular Division TCU Fort Worth, TX 76137 Fort Worth, TX 76129 [email protected] [email protected]

Abstract above. It is very difficult to see what is missing from a document by simply reading the document. Scenarios This work introduces a taxonomy of scenario use help one read and review with a different viewpoint with the objective to improve completeness of the because they encourage consideration of the details of requirements specification. Indications are given for each requirement instead of a general review. In future work to enable scenario development for some particular they enable discussion and review of dynamic weak categories of the taxonomy. behavior of the system under development. We are investigating scenarios as a means to improve completeness in a requirements specification document. Introduction Scenario Usage In systems development the first step in providing a solution that meets the customer’s needs is to define Scenario usage in development of software systems what problem must be solved. Requirements elicitation varies greatly and even the definition of a scenario is not and analysis is perhaps more iterative than any other consistent. Some use scenario to imply a particular phase of systems and software development. Rarely is customer use case, others refer to any sequence of system this process complete in one pass. Each iteration usually events as a scenario, and still others apply scenarios to includes these main activities: documenting the business goals or software inspections. We take the very requirements a system must provide; refining these general definition that a scenario refers to an ordered requirements; analysis for omissions, contradictions, and collection of inputs or internal states that produce a ambiguities; and review with consideration for a balance specific result. Ordered inputs might reflect a particular between customer concerns and desires, and profit and use case. Internal states implies, but does not timeliness of a solution. Requirements analysis often necessitate, state based analysis. An internal state could feeds the next iteration of requirements elicitation. illustrate an interface between two subsystems in which Completeness is one of the most difficult aspects to both subsystems can either communicate reliably or not assess in a requirements specification. Davis gives four communicate reliably. qualities of completeness in a requirements specification Basili and others have studied the use of scenarios in document: requirements inspections [PVB95]. However, careful 1 - All functionality required is included in the analysis of their use of scenarios shows their meaning document. differs from ours in this work. They define scenarios as 2 - Responses to all possible valid inputs and all “collections of procedures for detecting particular classes realizable invalid inputs are documented. of faults.” Their use of the term scenarios could perhaps 3 - All word processing is complete. This would also be interpreted as checklists; for example “Are all data include complete figures, tables of content, objects mentioned in the overview listed in the external required sections, and others. interface section?” is a scenario in their context. 4 - Nothing should be left to be determined. [Dav90] Several object-oriented methodologies, including the Scenarios aid in the detection of missing Unified Modeling Language (UML) methodology, requirements, addressing qualities one and two mentioned incorporate scenarios in the inception phase. In this phase, the scope and business case for the software is to create and use the scenarios during requirements analyzed. Use cases, as they are called in UML, help analysis. capture the requirements, stimulate discussion with the Ad hoc processes for generating scenarios and customer and users, and drive the tasks in design [UML]. analyzing requirements via these scenarios are the most A comprehensive study of the current use of prevalent. As ad hoc implies, there is no structured scenarios in software engineering and information approach to scenario definition, and use is inconsistent systems development was performed by Klausen and between practitioners. Generally the ad hoc process Aboulafia [KA94] . Their work focuses on the usage of consists of one or more developers recreating certain use scenarios in systems design and provides key insight into cases (i.e., human system interactions with an intended the engineers’ understanding and use of scenarios. They result) that have caused errors in operation in previous conducted interviews with several highly experienced systems or releases. These use cases or scenarios are engineers and categorized the responses. To give an idea then used to analyze new requirements or architectures. of the responses they encountered, one engineer indicated There is little or no recognition or documentation of that his company did not use scenarios or formal methods these scenarios. Klausen and Aboulafia determined the during design; however, they did use instances of user- process of creating and using scenarios is often ad hoc system interaction to spawn ideas and further design and unnamed. analysis. In addition, the end quality of the product was Holbrook has introduced Scenario Based strongly related to the personnel involved and the creation Requirements Elicitation (SBRE) [Hol90]. The stated of appropriate user-system interactions. Another purpose of the SBRE methodology is to “provide an respondent’s opinion was that performing scenario effective method for eliciting a user’s initial requirements analysis during design would take too much time. His based on the use of scenarios as a means of company’s typical product cycle is six months. communicating a design from designer to the user.” The Furthermore, he believes the scenarios generated are main goals of SBRE are to: typically unrealistic; however, his company employs 1 - Facilitate parallel development of the situations of use when evaluating early prototypes of the requirements and the high level design; product. 2 - Use scenarios to communicate the behavior of Analysis of the interviews gives some intuition into the design to the user; the current state of awareness of scenario usage for 3 - Use scenarios to assess the suitability of the software system development. Their results show: design; 1 - Awareness of the term “scenarios” is low; 4 - Create a collection of user’s issues which refine 2 - Scenarios are being used by different the scenarios and the design. organizations, though some organizations use The SBRE process creates scenarios in an iterative scenarios without calling them that; process where refinement is accomplished from feedback 3 - Any methodology formulated to create and given by the user. By considering the scenarios integrate scenarios must be easy to use and generated, the user can provide feedback each time the cannot be time intensive; functionality of the system is reviewed. Holbrook 4 - The quality and suitability of scenario selection stresses the importance of this scenario review; it can have a great impact on the quality of the end identifies unstated requirements and helps improve the product. requirements’ completeness. Finally, scenarios are used by software testers in Hsia and others have developed a methodology for integration and acceptance testing of functionally scenario generation to aid requirements elicitation structured or object-oriented based software [KGH95, [HSG94]. In this methodology, the scenario creation and TS93]. Often scenarios are directly translated into use process is combined with a prototype generation to system test cases. In addition, scenarios can be used for refine the scenarios and more closely match customer module testing by developers. Since the module desires with product functionality. To create scenarios, interfaces are known, it is relatively straightforward to they complete the following three steps: create scenarios that vary these inputs for testing each 1 - Identify and classify user groups. In this step, module. one logically groups all users of the software into one or more groups. Current Scenario Methodology 2 - Classify user views. User groups are identified as either active or dependent user views. In addition to reviewing current scenario usage, it is 3 - Construct scenario trees for each user view. For important to understand what methodologies are available each user view identified, a scenario tree is created which identifies system states and events. Each • Description: This describes how this category of node of the tree is a system state, and each event scenarios differs from all other categories. is identified as a transition from one system state • Creators/Users: This lists the architects and to another. developers most likely to create or use the scenario The scenarios are created by tracing events from the category. node at the top of the tree through a unique path to a • Information Needed: This lists the information terminal node on the bottom of the tree. Once all paths that is useful to create scenarios of this category. through the tree have been represented, scenario creation • Uses: This lists the most common uses of is complete. All the scenarios are logically associated scenarios of this category. with the user views, and a formal scenario grammar is In the taxonomy depicted in figure 1, scenarios can created for each user view. This leads to a thorough, describe basic functionality (operational), describe formal process for scenario creation, but the number of abnormal conditions (failure), help evaluate system scenarios generated can become quite large for most response (performance), aid in requirements analysis and systems. (Details on scenario trees and scenario elicitation (refinement), or be used to explain the system grammars can be found in [HSG94].) behavior to others (learning). Though each category and Our objective for studying scenario use is to help sub-category is unique, one scenario could be used in developers determine types of scenarios beneficial for different ways to allow it to be considered in more than analysis and especially to identify classes of scenarios one category. that are not created consistently by existing methodologies. Then ways to generate and utilize these Operational Scenarios scenarios are sought for use as part of the requirements elicitation and analysis process. An operational scenario (see table 1) describes some Scenarios currently are not categorized in any basic system functionality. Most scenarios that are used taxonomy, and distinctions between types of scenarios for requirements analysis would be classified as are not clear. A framework for discussing scenarios could operational scenarios. aid understanding and further discussion on the uses of scenarios. It could also uncover new classes of scenarios Table 1: Operational Scenarios that might help improve the completeness of a Description Scenarios used to evaluate the requirements specification process. system’s basic functionality. Creators/Users System Architects, Taxonomy of Scenarios Customers/Customer Liaison Information Needed System Inputs, Common Use A taxonomy of scenarios is presented to improve Cases understanding of scenarios and their usage. This Uses Requirements Verification and taxonomy of scenarios is organized into a hierarchy Validation, Architecture according to the purpose of the scenario. Though other Analysis hierarchies are possible, such as life-cycle phase, we have found this one to be useful for identifying types useful in Consider a project for development of a software requirements analysis. process database that will store defect metrics. The metrics in the database can be updated, and Phase Containment Effectiveness (PCE) and Phase Screening Scenario Effectiveness (PSE) reports can be generated to reflect the updates. Assume the database currently has valid defect data for some projects, say the Alpha and Bravo projects. For this example, generating PCE charts for the next Operational Failur e Performance Refi nement Learning release of the Bravo project would be an operational S cena rio Scenario Scenario Scenario Scenario scenario. Ad hoc approaches may result in operational scenarios. Hsia’s and Holbrook’s methods generate Figure 1: Hierarchy of Scenarios operational scenarios.

Each type of scenario is described in terms of key Failure Scenarios attributes: Failure scenarios describe abnormal behavior in a Table 3: Software Failure Scenarios system. In this context, abnormal behavior can be Description Scenarios which involve defined as behavior that would not be exhibited when the internal software errors. This user observes that the system is operating properly. could include inter-process Failure scenarios can be classified as ones caused by communication break-down, invalid input, internal faults, or faults in the internal invalid memory accesses, and interfaces between subsystems, as shown in figure 2. other general software faults. These are quite useful for analyzing requirements for a Creators/Users Requirements Inspectors, system that is to be tolerant of certain classes of faults. Customers/Customer Liaison Invalid input scenarios address behavior in the face of Information Needed Likely software faults, (based sequences or singular inputs that are not as specified (see on past history or defect table 2). Even if the system is not intended to be robust, analysis) i.e., to handle invalid input, these scenarios help identify Uses Requirements Analysis, likely problem areas. Customer Documentation

Failure Many means exist that allow multiple tasks to Scenario communicate. Message passing is one common means of inter-task communication. In such cases, queues can be used by each task to receive incoming messages. If

Invalid Input Softw are Failure Inter face Failure one task sends messages faster than the other can receive Scenario Scenario Scenario them, the receiving task may lose messages when the receiving task’s queue is full. A scenario that over-fills the receiver’s queue would be an example of a software failure scenario. Figure 2: Failure Scenario Hierarchy Table 4: Interface Failure Scenarios For an example of an invalid input scenario, consider Description Scenarios that involve the command line interface for a text based operating breakdown in system. The change directory command accepts one communication between input, the desired directory. An invalid input scenario subsystems separated by a would be one in which multiple parameters are entered physical or logical interface. with the command to change the directory. A command This breakdown could be to change directory to a non-existent directory is another because of a example. misinterpretation of an interface specification, a Table 2: Invalid Input Scenarios physical failure in the Description Scenarios which involve interface, or a suspect input that is outside allowed interface that cannot ranges, This could include transmit information human errors in using the reliably. user interface. Creators/Users System Architects, Field Creators/Users Requirements Inspectors, Engineers, Software Testers Designers Information Needed Valid ranges for all inputs Information Needed Physical Architecture Uses Requirements Analysis, Test showing most susceptible Generation interfaces Uses Requirements Verification Failure scenarios may also deal with defects in the and Validation, software or the interfaces between software components Requirements Elicitation, or software and hardware. Tables 3 and 4 describe these Test Case Generation categories. Consider two computers that share a common evaluation. Again, ad hoc approaches generate some interface such as a token ring. A collection of events performance scenarios. that cause the ring to be broken so that the two cannot communicate over that interface would be an interface Refinement Scenarios failure scenario. Ad hoc measures may generate failure scenarios to The context of use for a scenario will define whether evaluate behavior under certain conditions, but there is no it is a refinement scenario or some other type of scenario. focused methodology for this category. If the scenario is created with the intent to use only in requirements analysis, and it is not to be used in any Performance Scenarios other phase of the life-cycle, we call it a refinement scenario, as described in table 6. Performance scenarios are useful to discern additional capacity needed to meet performance constraints (see table Table 6: Refinement Scenarios 5). Combining scenarios with a simulation tool can Description Scenarios intended to aid in provide opportunities to estimate system performance the analysis of the software during system requirements. Suppose one wants to deliverable. decide how to set software parameters for a defined Creators/Users Requirements Specification network protocol, like TCP/IP, with the protocol Authors and Reviewers parameters P1, P2, and P3. Usually one would decide on Information Required Requirements Specification the single most important performance measure about the Uses Requirements Analysis system and optimize P1, P2, and P3 to yield the best measure for this metric. The most important Porter, Votta, and Basili address the use of scenarios performance measure might be minimizing the detection in requirements document inspections. But in such time for a broken communication link. Then the cases, scenarios are more of a checklist of known past following scenarios could be generated: areas of deficiencies. These scenarios typically are based The link is broken and parameter values on questions such as “Are all interfaces defined in the are P1 = 0, P2 = 1000, P2 = 3. The link Interface Document?” [PVB95] times out in S1 simulated time. Learning Scenarios The link is broken and parameter values are P1 = 10, P2 = 100000, P2 = 10. Scenarios are also used to illustrate ways in which The link times out in S2 simulated time. the system itself can be useful. These learning scenarios illustrate features and often typical user reactions and Table 5: Performance Scenarios inputs, as elaborated in table 7. Description Scenarios used to evaluate the system response or capacity. Table 7: Learning Scenarios Creators/Users System Architects, Description Scenarios intended to help Customers/Customer Liaison educate others about the Information Required Tunable Performance system features and behavior. Parameters, Desired Creators/Users System Architects, Software Performance Measure Engineers Uses Requirements Verification and Information Required Functionality crucial to Validation, Architecture others’ understanding of the Analysis, Hardware/Software system Partitioning Uses Training

By simulating a small series of these parameters, Soda vending machines are common textbook system performance can be analyzed and one can decide examples for software design methodologies. Several on optimal parameter settings. In this case, the two learning scenarios would exist, if the same system were scenarios above would be considered performance analyzed for requirements. One such scenario could be scenarios because they help with system response the user entering money to purchase a soda, classified as a learning scenario because it allows a new developer to understand the basics of coin collecting, coin counting, print text; additional features might include spell check and product disbursement. and thesaurus. Examples Edit keystrokes followed by mouse click cursor movement and additional edit This section provides additional examples to keystrokes; ending with the spell check differentiate between the different classifications of command keystroke scenario use that have been presented. Examples are is an operational scenario for studying this tool’s drawn from control systems, telephony, graphical user requirements. interface design, and requirements inspections. A final example scenario is from a client-server The first example involves a controller for a single architecture for a hospital computer network. elevator. PLookup, LName, FName The elevator is occupied, on the fourth floor of a ten story building, moving represents a client query to the server for a patient’s down; a patron pushes the up button on record. This is an operational scenario because it the second floor. describes normal operation. It is not difficult to imagine its use also as a learning scenario for engineers studying This is categorized as an operational scenario; it is a current system before proposing a new one. within normal operation. Next consider an in-building phone system that Commentary on Using the Taxonomy supports sixteen local phones. Only four digit extensions are accepted and a valid extension must start The taxonomy shows different types of scenarios with a digit from zero to eight. Valid inputs are: that are useful during the early phase(s) of system • Off hook (picking up the receiver) definition. It also provides a mechanism for exploring • On hook (hanging up the receiver) new methodologies for scenario creation. Methodologies • Not 9 (dialing a digit other than nine on the exist for generating operational scenarios. Perhaps one phone) or more scenario methodologies could be created to • Digit (dialing any digit (0-9) on the phone) address the other categories of scenarios identified in the • Busy (busy tone emitted from the telephone) taxonomy. Failure, performance, refinement, and Off hook, On hook learning scenarios tend to be distinct in function and different from operational ones. Even though existing is a typical scenario for a caller who begins to call methodologies might generate scenarios that are classified and then aborts. This scenario would be categorized as an as some of the above, these classifications do not seem operational scenario; it is a common, expected to be the focus of existing methodologies. It may be occurrence. difficult to develop one methodology that is designed to generate scenarios for each category; multiple techniques Off hook, Nine, Digit, Digit, Digit, On hook are needed. is categorized as an input failure scenario because An organized approach to creating these scenarios this sequence should not be entered during normal might also encourage adoption by some developers that operation of a telephone in this system. have not embraced existing scenario methodologies. Desirable characteristics for such an approach would Off hook, Not 9, Digit, Digit, Digit, Busy, include: On hook • Addressing some of the different (i.e., previously not considered) classes of scenarios identified by is an operational scenario because it represents the scenario taxonomy; normal operation when a caller dials a valid extension and • Keeping the number of scenarios generated to gets a busy signal. However in a different use necessary and sufficient levels while still playing environment, say one with auto voicemail default for an important role in requirements elicitation and busy phones, this scenario might be classified as a analysis; software or interface failure scenario. • Providing an organized step-by-step strategy; A graphical word processor provides a user-interface • Applying to both object-oriented and functional- example. Typical capabilities are enter, edit, save, and oriented approaches. Combining or refining existing methodologies, such References as Hsia’s, or Holbrook’s, may allow a more diverse set of scenarios. Perhaps experienced developers can use ad [Dav90] A. Davis, Software Requirements Analysis and hoc means to prune the scenario trees described by Hsia. Specification, Prentice-Hall, 1990. It seems likely that Holbrook’s SBRE approach could be [Hol90] H. Holbrook, “A Scenario-Based Methodology For refined to develop failure scenarios by involving the user Conducting Requirements Elicitation,” ACM SIGSOFT in dialog about undesired events. Or the user view of Software Engineering Notes, pp. 95-104, January 1990. Hsia’s approach could be broadened to address other [HSG94] P. Hsia, J. Samuel, J. Gao, and D. Kung, “Formal categories of scenario use. The ultimate goal of such Approach to Scenario Analysis”, IEEE Software, pp. 33 combining and expansion is to improve the completeness - 41, March 1994. of the requirement specification. [KA94] T. Klausen and A. Aboulafia, “An Empirical Study of The existing scenario methodologies often generate a Professional Software Designers’ Use of Scenarios,” large number of scenarios. Too many scenarios can make ESPRIT Basic Research Action 7040, November 1994. it difficult for one to incorporate these scenarios into [KGH95] D. Kung, J. Gao, P. Hsia, Y. Toyoshima, C. Chen, requirements elicitation and analysis which might lead Y. Kim, and Y. Song, “Developing an Object-Oriented some to reject a scenario methodology. Other Software Testing and Maintenance Environment”, methodologies are specialized for object-oriented analysis Communications of the ACM, vol. 38 (10), pp. 75-87, and design or graphical user interface design. This October 1995. [PHA89] M. Phadke, Quality Engineering Using Robust specialization makes widespread acceptance of such a Design, Prentice-Hall, 1989. methodology unlikely. [PVB95] A. Porter, L. Votta, and V. Basili, “Comparing We are studying techniques to limit the number of Detection Methods for Software Requirements scenario cases considered, while increasing the classes of Inspections: A Replicated Experiment,” IEEE scenarios considered. At present the robust design work Transactions on Software Engineering, vol. 21 no. 6, of Phadke [PHA89] for limiting test cases shows pp. 563 - 575, June 1995. promise when applied to requirements analysis scenarios [TS93] S. Tachimoto, S. Shiraishi, “Automatic Testing in the operational and failure categories, and will be System Based on Process Programming for reported in future papers. Telecommunication Software,” IEEE Global Telecommunications Conference, vol. 2, pp. 1017- 1021. [UML] http://www.rational.com/uml/index.shtml.