Software Quality Conformance to explicitly-stated functional and Software Quality Assurance and performance requirements, explicitly-documented Software Reuse development standards, and implicit characteristics that are expected of all professionally developed software. Peter Lo CS213 © Peter Lo 2005 1 CS213 © Peter Lo 2005 2 Three Important Points Quality Factors Product Operations: A software product's operational Quality will be measured with Software characteristics Requirements Product Revision: Its ability to undergo change Specified standards define a set of development Produced Transition: Its adaptability to new environments criteria that guide the manner in which software is engineered. The set of implicit requirements, such as good maintainability, must still be followed. CS213 © Peter Lo 2005 3 CS213 © Peter Lo 2005 4 Product Operations Product Operations Correctness Reliability The extent to which a program satisfies its The extent to which a program can be expected specification and fulfils the mission objectives. to perform its intended function with required Example: precision. Does it do what I want? Example: Does it do it accurately all the time? CS213 © Peter Lo 2005 5 CS213 © Peter Lo 2005 6 Product Operations Product Operations Efficiency Integrity The amount of computing resources and code The extent to which access to software or data required by a program to perform a function. by unauthorized persons can be controlled. Example: Example Will it run on my hardware as well as it can? Is it secure? CS213 © Peter Lo 2005 7 CS213 © Peter Lo 2005 8 Product Operations Product Revision Usability Maintainability The effort required to learn, operate, prepare The effort required to locate and fix an error in input, and interpret output of a program. a program. Example Example Is it designed for the user? Can I fix it? CS213 © Peter Lo 2005 9 CS213 © Peter Lo 2005 10 Product Revision Product Revision Flexibility Testability The effort required to modify an operational The effort required to test a program to ensure program. that it performs its intended function. Example Example Can I change it? Can I test it? CS213 © Peter Lo 2005 11 CS213 © Peter Lo 2005 12 Product Transition Product Transition Portability Reusability The effort to transfer the program from one The extent to which a program (or parts of a hardware and/or software system environment program) can be reused in other applications- to another. related to the packaging and scope of the Example functions that the program performs. Will I be able to use it on another machine? Example: Will I be able to reuse some of the software? CS213 © Peter Lo 2005 13 CS213 © Peter Lo 2005 14 Product Transition Software Quality Assurance Interoperability Software Quality Assurance (SQA) is a “planned The effort required to couple one system to and systematic pattern of actions” that are required another. to ensure quality in software. Example Will I be able to interface it with another system? CS213 © Peter Lo 2005 15 CS213 © Peter Lo 2005 16 SQA Activities SQA Activities Application of Technical Methods Conduct of Formal Technical Review Software Quality Assurance begins with a set Activity that accomplishes quality assessment of technical methods and tools that help the for the specification and the design. analyst to achieve a high-quality specification The Formal Technical Review is a stylized and the designer to develop a high-quality meeting conducted by technical staff with the design. sole purpose of uncovering quality problems. CS213 © Peter Lo 2005 17 CS213 © Peter Lo 2005 18 SQA Activities SQA Activities Testing of Software Enforcement of standards Combines a multi-step strategy with a series of Degree in which this is applied varies from test case design methods that help ensure company to company. effective error detection. In many cases, standards are dictated by customers or regulatory mandates. In other situations, standards are self-imposed. CS213 © Peter Lo 2005 19 CS213 © Peter Lo 2005 20 SQA Activities SQA Activities Control of Change Measurement Every change to software has the potential of Track software quality and assess the impact of introducing errors or creating side effects that methodological and procedural changes on propagate errors. improved software quality. The change control process thus contributes directly to software quality by formalizing requests for change, evaluating the nature of change, and controlling the impact of change. CS213 © Peter Lo 2005 21 CS213 © Peter Lo 2005 22 SQA Activities Formal Technical Reviews Record keeping and recording Reviews are applied at various points during Collection and dissemination of SQA software development and serve to uncover errors information. that can then be removed. The results of reviews, audits, change control, Formal technical review is a class of reviews that testing, and other SQA activities must become include walkthroughs, inspections, round-robin part of a historical record for a project and reviews, and other small group technical should be disseminated to the development staff assessments of software. on a need-to-know basis. It is a planned and controlled meeting attended by a group of diversified people. CS213 © Peter Lo 2005 23 CS213 © Peter Lo 2005 24 Objectives of Formal Technical Purposes of Reviews Review (FTR) Point out need improvements in the product of a To uncover errors in function, logic, or implementation for single person or team. any representation of the software To verify that the software under review meets its Confirm those parts of a product in which requirements improvement is either not desired or not needed. To ensure that the software has been represented according Achieve technical work of more uniform, or at to predefined standards least more predictable, quality than can be To achieve software that is developed in a uniform manner achieved without reviews, in order to make To make projects more manageable technical work more manageable. CS213 © Peter Lo 2005 25 CS213 © Peter Lo 2005 26 Guidelines for the Organization Guidelines for the Organization and Preparation of FTR and Preparation of FTR Should involve between three and five people A recorder should actively record all issues as Advance preparation should occur but should not require What was reviewed? more than 2 hours of work for each person Who reviewed it? The duration of the review meeting should be less than 2 What were the findings and conclusions? hours The attendees must make a decision at the end of the session as to Focus on a components of the software Accept the product e.g. a portion of requirements specification, a detailed module design, a source code listing for a module Reject the product Accept the product provisionally, subject to further Producer should report his/her progress modification CS213 © Peter Lo 2005 27 CS213 © Peter Lo 2005 28 Review Guidelines during the FTR Review Guidelines during the FTR Review the product, not the producer Develop a checklist for each product that is likely Set an agenda and maintain it to be reviewed Limit debate and rebuttal Allocate resources and time schedule for Formal Technical Reviews Enunciate problem areas, but don't attempt to solve every problem noted Conduct meaningful training for all reviewers Take written notes Review your early reviews Limit the number of participants and insist upon advance preparation CS213 © Peter Lo 2005 29 CS213 © Peter Lo 2005 30 Software Reliability Types of Failure Software Reliability is the probability of failure Transient free operation of a computer program in a Occasional, or only with certain inputs. specified environment for a specified time. Permanent Software Availability is the probability that a Occurs with all inputs. program is operating according to requirements at a given point in time. Recoverable Allows system to continue operation without operator intervention. CS213 © Peter Lo 2005 31 CS213 © Peter Lo 2005 32 Types of Failure Tradeoff between Cost and Reliability Unrecoverable It demands more time and resources for testing Forces system to halt operation. and debugging. Non-corrupting It requires more internal safety checks, and this makes programs larger and slower. Doesn't destroy data. Corrupting Does destroy data. CS213 © Peter Lo 2005 33 CS213 © Peter Lo 2005 34 Reliability Metric Mean Time Between Failure (MTBF) Reliability metric is an indicator of how broken a MTBF = MTTF + MTTR program is. Measures how long a program is likely to run Metrics are best weighted by the by severity of before it does something bad like crash. errors. MTTF is the mean time to failure. A minor error every hour is better than a MTTR is the mean time to repair. catastrophe every month. These metrics are not the same as counting bugs, but indicate different probabilities of happening. CS213 © Peter Lo 2005 35 CS213 © Peter Lo 2005 36 Availability Probability of Failure on Demand Availability = MTTF/ (MTTF + MTTR) * 100% Common for safety-critical systems MTTF is the mean time to failure. e.g. a specification may be that there not exceed 10 MTTR is the mean time to repair. 1 chance in 10 that a missile gets through. CS213 © Peter Lo 2005 37 CS213 © Peter Lo 2005 38 Rate of Failure Occurrence Software Quality Assurance Approach This tells how many may occur in a given period At the low end of the scale, quality is the sole e.g. 2 errors per 100 minutes.