Software Reliability and Security S Yashwant K
Total Page:16
File Type:pdf, Size:1020Kb
Software Reliability and Security S Yashwant K. Malaiya Computer Science Department, Colorado State University, Fort Collins, Colorado, U.S.A. INTRODUCTION in achieving and evaluating reliability. Here, we will use the terms ‘‘failure’’ and ‘‘defect’’ as defined below.[1] Software problems are the main causes of system Failure: A departure of the system behavior from failures today. There are many well-known cases of user requirements during execution. the tragic consequences of software failures. In critical Defect (fault or bug): An error in system implemen- systems, very high reliability is naturally expected. tation that can cause a failure during execution. Software packages used everyday also need to be A defect will cause a failure only when the erro- highly reliable, because the enormous investment of neous code is executed, and the effect is propagated the software developer is at stake. Studies have shown to the output. The testability of a defect is defined as that reliability is regarded as the most important attri- the probability of detecting it with a randomly chosen bute by potential customers. All software developed input. Defects with very low testability can be very dif- will have a significant number of defects. All programs ficult to detect. The software reliability improves dur- must be tested and debugged, until sufficiently high ing testing, as bugs are found and removed. Once the reliability is achieved. It is not possible to ensure that software is released, its reliability is fixed as long as all the defects in a software system have been found the operating environment remains the same. The soft- and removed; however, the number of remaining bugs ware will fail time to time during operational use when must be very small. As software must be released it cannot respond correctly to an input. During opera- within a reasonable time, to avoid loss of revenue tional use, bug fixes that update the software are often and market share, the developer must take a calculated released. For a software system, its own past behavior risk and must have a strategy for achieving the is often a good indicator of its reliability, even though required reliability by the target release date. For soft- data from other similar software systems can be used ware systems, quantitative methods for achieving and for making projections.[2] measuring reliability are coming in use because of the emergence of well-understood and validated approaches. Enough industrial and experimental data RELIABILITY APPROACHES DURING THE are available to develop and validate methods for SOFTWARE LIFE CYCLE PHASES achieving high reliability. The minimum acceptable standards for software reliability have gradually risen A competitive and mature software development orga- in recent years. nization targets a high reliability objective from the This entry presents an overview of the essential con- very beginning of software development. Generally, cepts and techniques in the software reliability field. the software life cycle is divided into the following We examine factors that impact reliability during phases. It is not uncommon for developers to go to a development as well as during testing. First, we discuss previous phase, it may become necessary because of the reliability approaches taken during different phases the requirement changes or a need to make changes of software development. Integration and interoper- in the design. It is preferable to catch defects early as ability testing are examined. Commonly used software it would be more expensive to fix them later. reliability measures are defined next. We discuss what factors control software defect density. Key ideas in 1. Requirements and definition: In this phase, the test methodologies are presented. Use of a software developing organization interacts with the customer reliability growth model is discussed and illustrated organization to specify the software system to be built. using industrial data. Use of such models allows one Ideally, the requirements should define the system to estimate the testing effort needed to reach a reliabil- completely and unambiguously. In actual practice, ity goal. We also discuss a similar model for security there is often a need to do corrective revisions during vulnerabilities in internet-related software. We also software development. A review or inspection during see how reliability of a system can be evaluated if we this phase is generally done by the design team to iden- know the failure rates of the components. Finally, the arti- tify conflicting or missing requirements. A significant cle presents the type of tools that are available to assist number of errors can be detected by this process. Encyclopedia of Library and Information Science DOI: 10.1081/E-ELIS-120034145 Copyright # 2005 by Taylor & Francis. All rights reserved. 1 2 Software Reliability and Security A change in the requirements in the later phases can The time taken to develop a patch after a vulnerability cause increased defect density. discovery, and the delayed application of an available patch contribute to the security risks. When significant 2. Design: In this phase, the system is specified as additions or modifications are made to an existing ver- an interconnection of units, such that each unit is well sion, regression testing is done on the new or ‘‘build’’ ver- defined and can be developed and tested indepen- sion to ensure that it still works and has not ‘‘regressed’’ dently. The design is reviewed to recognize errors. to lower reliability. Support for an older version of a soft- 3. Coding: In this phase, the actual program for ware product needs to be offered until newer versions each unit is written, generally in a higher-level lan- have made a prior version relatively obsolete. guage such as Java or Cþþ. Occasionally, assembly It should be noted that the exact definition of a test level implementation may be required for high perfor- phase and its exit criterion may vary from organization mance or for implementing input=output operations. to organization. When a project goes through incre- The code is inspected by analyzing the code (or speci- mental refinements (as in the extreme programming fication) in a team meeting to identify errors. approach), there may be many cycles of requirements– design–code–test phases. 4. Testing: This phase is a critical part of the quest Table 1 shows the typical fraction of total defects for high reliability and can take 30–60% of the entire introduced and found during a phase.[3,4] Most defects development time. It is often divided into the following occur during the design and coding phases. The frac- separate phases. tion of defects found during the system test is small, but that may be misleading. The system test phase a. Unit test: In this phase of testing, each unit is can take a long time because the defects remaining separately tested, and changes are done to are much harder to find. It has been observed that remove the defects found. As each unit is rela- the testing phases can account for 30–60% of the entire tively small and can be tested independently, development effort. they can be exercised much more thoroughly Two types of testing deserve special attention: inte- than a large program. gration testing and interoperability testing. Integration b. Integration testing: During integration, the testing assumes that unit testing has already been done, units are gradually assembled and partially and thus the focus is on testing for defects that are assembled subsystems are tested. Testing sub- associated with interaction among the modules. Exer- systems allows the interface among modules to cising a unit module requires use of a driver module, be tested. By incrementally adding units to a which will call the unit under test. If a unit module subsystem, the unit responsible for a failure does not call other units, it is called a terminal module. can be identified more easily. If a unit module calls other modules, which are not yet c. System testing: The system as a whole is exer- ready to be used, surrogate modules called stubs simu- cised during system testing. Debugging is con- late the interaction. tinued until some exit criterion is satisfied. The Integration testing can be bottom-up or top-down. objective of this phase is to find defects as fast In the bottom-up approach, integration starts with as possible. In general, the input mix may not attaching the terminal modules to the modules that call represent what would be encountered during them. This requires the use of drivers to drive the the actual operation. higher-level modules. The top-down integration starts d. Acceptance testing: The purpose of this test with connecting the highest-level modules with the phase is to assess the system reliability and per- formance in the operational environment. This requires collecting (or estimating) information Table 1 Defects introduced and found during on how the actual users would use the system. different phases This is also called a-testing. This is often fol- Defects (%) lowed by b-testing, which involves use of the b-version by the actual users. Phase Introduced Found Remaining Requirements 10 5 5 5. Operational use and maintenance: Once the analysis software developer has determined that an appropriate Design 35 15 25 reliability criterion is satisfied, the software is released. Coding 45 30 40 Any bugs reported by the users are recorded but are Unit test 5 25 20 not fixed until the next patch or bug-fix. In case a Integration test 2 12 10 defect discovered represents a security vulnerability, a System test 1 10 1 patch for it needs to be released as soon as possible. Software Reliability and Security 3 modules called by them. This requires the use of stubs 2. Transaction reliability: Sometimes a single- until finally the terminal modules are integrated.