Conformance to explicitly-stated functional and Software Quality Assurance and performance requirements, explicitly-documented Software Reuse development standards, and implicit characteristics that are expected of all professionally developed software. Peter Lo
CS213 © Peter Lo 2005 1 CS213 © Peter Lo 2005 2
Three Important Points Quality Factors
Product Operations: A software product's operational Quality will be measured with Software characteristics Requirements Product Revision: Its ability to undergo change Specified standards define a set of development Produced Transition: Its adaptability to new environments criteria that guide the manner in which software is engineered. The set of implicit requirements, such as good maintainability, must still be followed.
CS213 © Peter Lo 2005 3 CS213 © Peter Lo 2005 4 Product Operations Product Operations
Correctness Reliability
The extent to which a program satisfies its The extent to which a program can be expected specification and fulfils the mission objectives. to perform its intended function with required
Example: precision.
Does it do what I want? Example: Does it do it accurately all the time?
CS213 © Peter Lo 2005 5 CS213 © Peter Lo 2005 6
Product Operations Product Operations
Efficiency Integrity
The amount of computing resources and code The extent to which access to software or data required by a program to perform a function. by unauthorized persons can be controlled.
Example: Example
Will it run on my hardware as well as it can? Is it secure?
CS213 © Peter Lo 2005 7 CS213 © Peter Lo 2005 8 Product Operations Product Revision
Usability Maintainability
The effort required to learn, operate, prepare The effort required to locate and fix an error in input, and interpret output of a program. a program.
Example Example
Is it designed for the user? Can I fix it?
CS213 © Peter Lo 2005 9 CS213 © Peter Lo 2005 10
Product Revision Product Revision
Flexibility Testability
The effort required to modify an operational The effort required to test a program to ensure program. that it performs its intended function.
Example Example
Can I change it? Can I test it?
CS213 © Peter Lo 2005 11 CS213 © Peter Lo 2005 12 Product Transition Product Transition
Portability Reusability
The effort to transfer the program from one The extent to which a program (or parts of a hardware and/or software system environment program) can be reused in other applications- to another. related to the packaging and scope of the
Example functions that the program performs.
Will I be able to use it on another machine? Example: Will I be able to reuse some of the software?
CS213 © Peter Lo 2005 13 CS213 © Peter Lo 2005 14
Product Transition Software Quality Assurance
Interoperability Software Quality Assurance (SQA) is a “planned
The effort required to couple one system to and systematic pattern of actions” that are required another. to ensure quality in software.
Example
Will I be able to interface it with another system?
CS213 © Peter Lo 2005 15 CS213 © Peter Lo 2005 16 SQA Activities SQA Activities
Application of Technical Methods Conduct of Formal Technical Review
Software Quality Assurance begins with a set Activity that accomplishes quality assessment of technical methods and tools that help the for the specification and the design.
analyst to achieve a high-quality specification The Formal Technical Review is a stylized and the designer to develop a high-quality meeting conducted by technical staff with the design. sole purpose of uncovering quality problems.
CS213 © Peter Lo 2005 17 CS213 © Peter Lo 2005 18
SQA Activities SQA Activities
Testing of Software Enforcement of standards
Combines a multi-step strategy with a series of Degree in which this is applied varies from test case design methods that help ensure company to company.
effective error detection. In many cases, standards are dictated by customers or regulatory mandates. In other situations, standards are self-imposed.
CS213 © Peter Lo 2005 19 CS213 © Peter Lo 2005 20 SQA Activities SQA Activities
Control of Change Measurement Every change to software has the potential of Track software quality and assess the impact of introducing errors or creating side effects that methodological and procedural changes on propagate errors. improved software quality. The change control process thus contributes directly to software quality by formalizing requests for change, evaluating the nature of change, and controlling the impact of change.
CS213 © Peter Lo 2005 21 CS213 © Peter Lo 2005 22
SQA Activities Formal Technical Reviews
Record keeping and recording Reviews are applied at various points during
Collection and dissemination of SQA software development and serve to uncover errors information. that can then be removed.
The results of reviews, audits, change control, Formal technical review is a class of reviews that testing, and other SQA activities must become include walkthroughs, inspections, round-robin part of a historical record for a project and reviews, and other small group technical should be disseminated to the development staff assessments of software. on a need-to-know basis. It is a planned and controlled meeting attended by a group of diversified people.
CS213 © Peter Lo 2005 23 CS213 © Peter Lo 2005 24 Objectives of Formal Technical Purposes of Reviews Review (FTR)
Point out need improvements in the product of a To uncover errors in function, logic, or implementation for single person or team. any representation of the software To verify that the software under review meets its Confirm those parts of a product in which requirements improvement is either not desired or not needed. To ensure that the software has been represented according Achieve technical work of more uniform, or at to predefined standards least more predictable, quality than can be To achieve software that is developed in a uniform manner achieved without reviews, in order to make To make projects more manageable technical work more manageable.
CS213 © Peter Lo 2005 25 CS213 © Peter Lo 2005 26
Guidelines for the Organization Guidelines for the Organization and Preparation of FTR and Preparation of FTR
Should involve between three and five people A recorder should actively record all issues as Advance preparation should occur but should not require What was reviewed? more than 2 hours of work for each person Who reviewed it? The duration of the review meeting should be less than 2 What were the findings and conclusions? hours The attendees must make a decision at the end of the session as to Focus on a components of the software Accept the product e.g. a portion of requirements specification, a detailed module design, a source code listing for a module Reject the product Accept the product provisionally, subject to further Producer should report his/her progress modification
CS213 © Peter Lo 2005 27 CS213 © Peter Lo 2005 28 Review Guidelines during the FTR Review Guidelines during the FTR
Review the product, not the producer Develop a checklist for each product that is likely Set an agenda and maintain it to be reviewed Limit debate and rebuttal Allocate resources and time schedule for Formal Technical Reviews Enunciate problem areas, but don't attempt to solve every problem noted Conduct meaningful training for all reviewers Take written notes Review your early reviews Limit the number of participants and insist upon advance preparation
CS213 © Peter Lo 2005 29 CS213 © Peter Lo 2005 30
Software Reliability Types of Failure
Software Reliability is the probability of failure Transient
free operation of a computer program in a Occasional, or only with certain inputs. specified environment for a specified time. Permanent Software Availability is the probability that a Occurs with all inputs. program is operating according to requirements at a given point in time. Recoverable Allows system to continue operation without operator intervention.
CS213 © Peter Lo 2005 31 CS213 © Peter Lo 2005 32 Types of Failure Tradeoff between Cost and Reliability
Unrecoverable It demands more time and resources for testing
Forces system to halt operation. and debugging. Non-corrupting It requires more internal safety checks, and this makes programs larger and slower. Doesn't destroy data. Corrupting
Does destroy data.
CS213 © Peter Lo 2005 33 CS213 © Peter Lo 2005 34
Reliability Metric Mean Time Between Failure (MTBF)
Reliability metric is an indicator of how broken a MTBF = MTTF + MTTR program is. Measures how long a program is likely to run Metrics are best weighted by the by severity of before it does something bad like crash.
errors. MTTF is the mean time to failure.
A minor error every hour is better than a MTTR is the mean time to repair. catastrophe every month. These metrics are not the same as counting bugs, but indicate different probabilities of happening.
CS213 © Peter Lo 2005 35 CS213 © Peter Lo 2005 36 Availability Probability of Failure on Demand
Availability = MTTF/ (MTTF + MTTR) * 100% Common for safety-critical systems
MTTF is the mean time to failure. e.g. a specification may be that there not exceed 10 MTTR is the mean time to repair. 1 chance in 10 that a missile gets through.
CS213 © Peter Lo 2005 37 CS213 © Peter Lo 2005 38
Rate of Failure Occurrence Software Quality Assurance Approach
This tells how many may occur in a given period At the low end of the scale, quality is the sole
e.g. 2 errors per 100 minutes. This is considered responsibility of the individual who may engineer, one of the best metrics review, and test at any comfort level. At the high end of the scale, an SQA group is charged with the responsibility standards and procedures for achieving software quality and ensuring that each is followed.
CS213 © Peter Lo 2005 39 CS213 © Peter Lo 2005 40 Benefits of Software Quality Constraints of Software Quality Assurance Assurance
Software will have fewer latent defects, resulting Difficult to institute in small organizations, where in reduced effort and time spent during testing and available resources to perform the necessary maintenance activities are not available
Higher reliability will result in greater customer Software Quality Assurance represents cultural satisfaction change - and change is never easy
Maintenance costs Software Quality Assurance required the
Overall life cycle cost of software is reduced expenditure of dollars that would not otherwise be explicitly budgeted to software engineering or quality assurance
CS213 © Peter Lo 2005 41 CS213 © Peter Lo 2005 42
Software Reuse Management Issues in Software Reuse
Modern, complex, high-quality computer-based Few companies and organizations have anything that even slightly resembles a comprehensive software reusability systems must be built in very short time periods, plan. and this demands a more organized approach to Relatively little training is available to help software software reuse. engineers and managers understand and apply reuse. Many software practitioners continue to believe that reuse is “more trouble than it is worth.” Many companies continue to encourage the use of methodologies that do not facilitate reuse Few companies provide incentives to produce reusable program components.
CS213 © Peter Lo 2005 43 CS213 © Peter Lo 2005 44 Reusable Artifacts Impact of Reuse on Quality
Project Plans Software component that is developed for reuse Cost estimates would contain defects. Requirements models and specifications These defects like these can be found and eliminated, and the reused component quality Source code improves. User and technical documentation Over time, the component becomes virtually Human interfaces defect free. Data: e.g. tables, lists, record structures Test cases
CS213 © Peter Lo 2005 45 CS213 © Peter Lo 2005 46
Impact of Reuse on Productivity Impact of Reuse on Cost
When reusable artifacts are applied throughout the The net savings for reuse can be estimated by the software process, less time is spent creating the following:
plans, models, documents, code and data that are Net Savings = (Cost of the project developed normally required to create a deliverable system. from scratch) – (sum of costs associated with The same level of functionality can be delivered to reuse) – (Actual cost of the software as the customer with less input effort. delivered)
CS213 © Peter Lo 2005 47 CS213 © Peter Lo 2005 48