„ Conformance to explicitly-stated functional and Software Quality Assurance and performance requirements, explicitly-documented Software Reuse development standards, and implicit characteristics that are expected of all professionally developed software. Peter Lo

CS213 © Peter Lo 2005 1 CS213 © Peter Lo 2005 2

Three Important Points Quality Factors

„ Product Operations: A software product's operational „ Quality will be measured with Software characteristics Requirements „ Product Revision: Its ability to undergo change „ Specified standards define a set of development „ Produced Transition: Its to new environments criteria that guide the manner in which software is engineered. „ The set of implicit requirements, such as good , must still be followed.

CS213 © Peter Lo 2005 3 CS213 © Peter Lo 2005 4 Product Operations Product Operations

„ Correctness „ Reliability

‹ The extent to which a program satisfies its ‹ The extent to which a program can be expected specification and fulfils the mission objectives. to perform its intended function with required

‹ Example: precision.

 Does it do what I want? ‹ Example:  Does it do it accurately all the time?

CS213 © Peter Lo 2005 5 CS213 © Peter Lo 2005 6

Product Operations Product Operations

„ Efficiency „ Integrity

‹ The amount of computing resources and code ‹ The extent to which access to software or data required by a program to perform a function. by unauthorized persons can be controlled.

‹ Example: ‹ Example

 Will it run on my hardware as well as it can?  Is it secure?

CS213 © Peter Lo 2005 7 CS213 © Peter Lo 2005 8 Product Operations Product Revision

„ „ Maintainability

‹ The effort required to learn, operate, prepare ‹ The effort required to locate and fix an error in input, and interpret output of a program. a program.

‹ Example ‹ Example

 Is it designed for the user?  Can I fix it?

CS213 © Peter Lo 2005 9 CS213 © Peter Lo 2005 10

Product Revision Product Revision

„ Flexibility „ Testability

‹ The effort required to modify an operational ‹ The effort required to test a program to ensure program. that it performs its intended function.

‹ Example ‹ Example

 Can I change it?  Can I test it?

CS213 © Peter Lo 2005 11 CS213 © Peter Lo 2005 12 Product Transition Product Transition

„ Portability „ Reusability

‹ The effort to transfer the program from one ‹ The extent to which a program (or parts of a hardware and/or software system environment program) can be reused in other applications- to another. related to the packaging and scope of the

‹ Example functions that the program performs.

 Will I be able to use it on another machine? ‹ Example:  Will I be able to reuse some of the software?

CS213 © Peter Lo 2005 13 CS213 © Peter Lo 2005 14

Product Transition Software Quality Assurance

„ Interoperability „ Software Quality Assurance (SQA) is a “planned

‹ The effort required to couple one system to and systematic pattern of actions” that are required another. to ensure quality in software.

‹ Example

 Will I be able to interface it with another system?

CS213 © Peter Lo 2005 15 CS213 © Peter Lo 2005 16 SQA Activities SQA Activities

„ Application of Technical Methods „ Conduct of Formal Technical Review

‹ Software Quality Assurance begins with a set ‹ Activity that accomplishes quality assessment of technical methods and tools that help the for the specification and the design.

analyst to achieve a high-quality specification ‹ The Formal Technical Review is a stylized and the designer to develop a high-quality meeting conducted by technical staff with the design. sole purpose of uncovering quality problems.

CS213 © Peter Lo 2005 17 CS213 © Peter Lo 2005 18

SQA Activities SQA Activities

„ Testing of Software „ Enforcement of standards

‹ Combines a multi-step strategy with a series of ‹ Degree in which this is applied varies from test case design methods that help ensure company to company.

effective error detection. ‹ In many cases, standards are dictated by customers or regulatory mandates. In other situations, standards are self-imposed.

CS213 © Peter Lo 2005 19 CS213 © Peter Lo 2005 20 SQA Activities SQA Activities

„ Control of Change „ Measurement ‹ Every change to software has the potential of ‹ Track software quality and assess the impact of introducing errors or creating side effects that methodological and procedural changes on propagate errors. improved software quality. ‹ The change control process thus contributes directly to software quality by formalizing requests for change, evaluating the nature of change, and controlling the impact of change.

CS213 © Peter Lo 2005 21 CS213 © Peter Lo 2005 22

SQA Activities Formal Technical Reviews

„ Record keeping and recording „ Reviews are applied at various points during

‹ Collection and dissemination of SQA and serve to uncover errors information. that can then be removed.

‹ The results of reviews, audits, change control, „ Formal technical review is a class of reviews that testing, and other SQA activities must become include walkthroughs, inspections, round-robin part of a historical record for a project and reviews, and other small group technical should be disseminated to the development staff assessments of software. on a need-to-know basis. „ It is a planned and controlled meeting attended by a group of diversified people.

CS213 © Peter Lo 2005 23 CS213 © Peter Lo 2005 24 Objectives of Formal Technical Purposes of Reviews Review (FTR)

„ Point out need improvements in the product of a „ To uncover errors in function, logic, or implementation for single person or team. any representation of the software „ To verify that the software under review meets its „ Confirm those parts of a product in which requirements improvement is either not desired or not needed. „ To ensure that the software has been represented according „ Achieve technical work of more uniform, or at to predefined standards least more predictable, quality than can be „ To achieve software that is developed in a uniform manner achieved without reviews, in order to make „ To make projects more manageable technical work more manageable.

CS213 © Peter Lo 2005 25 CS213 © Peter Lo 2005 26

Guidelines for the Organization Guidelines for the Organization and Preparation of FTR and Preparation of FTR

„ Should involve between three and five people „ A recorder should actively record all issues as „ Advance preparation should occur but should not require ‹ What was reviewed? more than 2 hours of work for each person ‹ Who reviewed it? „ The duration of the review meeting should be less than 2 ‹ What were the findings and conclusions? hours „ The attendees must make a decision at the end of the session as to „ Focus on a components of the software ‹ Accept the product ‹ e.g. a portion of requirements specification, a detailed module design, a source code listing for a module ‹ Reject the product ‹ Accept the product provisionally, subject to further „ Producer should report his/her progress modification

CS213 © Peter Lo 2005 27 CS213 © Peter Lo 2005 28 Review Guidelines during the FTR Review Guidelines during the FTR

„ Review the product, not the producer „ Develop a checklist for each product that is likely „ Set an agenda and maintain it to be reviewed „ Limit debate and rebuttal „ Allocate resources and time schedule for Formal Technical Reviews „ Enunciate problem areas, but don't attempt to solve every problem noted „ Conduct meaningful training for all reviewers „ Take written notes „ Review your early reviews „ Limit the number of participants and insist upon advance preparation

CS213 © Peter Lo 2005 29 CS213 © Peter Lo 2005 30

Software Reliability Types of Failure

„ Software Reliability is the probability of failure „ Transient

free operation of a computer program in a ‹ Occasional, or only with certain inputs. specified environment for a specified time. „ Permanent „ Software Availability is the probability that a ‹ Occurs with all inputs. program is operating according to requirements at a given point in time. „ Recoverable ‹ Allows system to continue operation without operator intervention.

CS213 © Peter Lo 2005 31 CS213 © Peter Lo 2005 32 Types of Failure Tradeoff between Cost and Reliability

„ Unrecoverable „ It demands more time and resources for testing

‹ Forces system to halt operation. and debugging. „ Non-corrupting „ It requires more internal safety checks, and this makes programs larger and slower. ‹ Doesn't destroy data. „ Corrupting

‹ Does destroy data.

CS213 © Peter Lo 2005 33 CS213 © Peter Lo 2005 34

Reliability Metric Mean Time Between Failure (MTBF)

„ Reliability metric is an indicator of how broken a „ MTBF = MTTF + MTTR program is. „ Measures how long a program is likely to run „ Metrics are best weighted by the by severity of before it does something bad like crash.

errors. ‹ MTTF is the mean time to failure.

„ A minor error every hour is better than a ‹ MTTR is the mean time to repair. catastrophe every month. „ These metrics are not the same as counting bugs, but indicate different probabilities of happening.

CS213 © Peter Lo 2005 35 CS213 © Peter Lo 2005 36 Availability Probability of Failure on Demand

„ Availability = MTTF/ (MTTF + MTTR) * 100% „ Common for safety-critical systems

‹ MTTF is the mean time to failure. ‹ e.g. a specification may be that there not exceed 10 ‹ MTTR is the mean time to repair. 1 chance in 10 that a missile gets through.

CS213 © Peter Lo 2005 37 CS213 © Peter Lo 2005 38

Rate of Failure Occurrence Software Quality Assurance Approach

„ This tells how many may occur in a given period „ At the low end of the scale, quality is the sole

‹ e.g. 2 errors per 100 minutes. This is considered responsibility of the individual who may engineer, one of the best metrics review, and test at any comfort level. „ At the high end of the scale, an SQA group is charged with the responsibility standards and procedures for achieving software quality and ensuring that each is followed.

CS213 © Peter Lo 2005 39 CS213 © Peter Lo 2005 40 Benefits of Software Quality Constraints of Software Quality Assurance Assurance

„ Software will have fewer latent defects, resulting „ Difficult to institute in small organizations, where in reduced effort and time spent during testing and available resources to perform the necessary maintenance activities are not available

‹ Higher reliability will result in greater customer „ Software Quality Assurance represents cultural satisfaction change - and change is never easy

‹ Maintenance costs „ Software Quality Assurance required the

‹ Overall life cycle cost of software is reduced expenditure of dollars that would not otherwise be explicitly budgeted to or quality assurance

CS213 © Peter Lo 2005 41 CS213 © Peter Lo 2005 42

Software Reuse Management Issues in Software Reuse

„ Modern, complex, high-quality computer-based „ Few companies and organizations have anything that even slightly resembles a comprehensive software reusability systems must be built in very short time periods, plan. and this demands a more organized approach to „ Relatively little training is available to help software software reuse. engineers and managers understand and apply reuse. „ Many software practitioners continue to believe that reuse is “more trouble than it is worth.” „ Many companies continue to encourage the use of methodologies that do not facilitate reuse „ Few companies provide incentives to produce reusable program components.

CS213 © Peter Lo 2005 43 CS213 © Peter Lo 2005 44 Reusable Artifacts Impact of Reuse on Quality

„ Project Plans „ Software component that is developed for reuse „ Cost estimates would contain defects. „ Requirements models and specifications „ These defects like these can be found and eliminated, and the reused component quality „ Source code improves. „ User and technical documentation „ Over time, the component becomes virtually „ Human interfaces defect free. „ Data: e.g. tables, lists, record structures „ Test cases

CS213 © Peter Lo 2005 45 CS213 © Peter Lo 2005 46

Impact of Reuse on Productivity Impact of Reuse on Cost

„ When reusable artifacts are applied throughout the „ The net savings for reuse can be estimated by the software process, less time is spent creating the following:

plans, models, documents, code and data that are ‹ Net Savings = (Cost of the project developed normally required to create a deliverable system. from scratch) – (sum of costs associated with „ The same level of functionality can be delivered to reuse) – (Actual cost of the software as the customer with less input effort. delivered)

CS213 © Peter Lo 2005 47 CS213 © Peter Lo 2005 48