Tool Support for Software Quality Assurance L

Tool support for software quality assurance L. Fernandez Sanz

Departmento de Lenguajes y Sistemas fm/ormaZzcos e 7%#emerm de/ ^o/^are de Informdtica, Universidad Politecnica de

Madrid, Spain

Abstract

This paper provides an analysis of two important factors for introducing project Software Quality Assurance (SQA) activities tool support: tool selection and availability of SQA tools (software tools). Firstly, SQA activities during software development projects are reviewed under the perspective of Software

Engineering standards. The paper uses these activities as a basis for providing a method to establish an adequate SQA tools introduction strategy that can be applied to any software development organization. Proposed tool selection approach is an attempt to extend author's CASE selection method based on the

Capability Maturity Model (CMM). Finally, the paper discusses the availability of appropiate SQA tools. Tools panorama often leads software professionals to look for alternative sources (apart from marketed tools) to obtain tools that can support organization's SQA activities. Several industrial experiences are described in which these alternative sources of tools have been succesfully applied to support review techniques and software metrics.

1 Introduction to Software Quality Assurance

Quality seems to be the star phenomenon at the end of the century in all productive sectors. Experts indicate that quality is one of the main competitive advantages for products and services. This new important role of quality has been perceived through new management scopes. Apart from the two traditional management objectives (money and time), now quality is another point of interest for business managers. Software development and maintenance is not an exception, especially when the expression "software crisis" (high maintenance cost, low productivity and continued poor quality) is often heard. Besides quality can be profitable, as stated in several references (e.g. see

634 Software Quality Management

Fernandez et al.[2], Vincent et al.[3], Kitchenham [4]).

Software quality management has adapted ideas from the manufacturing area to provide a way for structuring organization's work methods to achieve quality objectives. This approach often leads to an organizational Quality Management System (QMS) with the purpose of providing "the organizational structure, responsibilities, procedures, processes and resources for implementing quality management"(ISO [5]).

Although organization's quality management is a basic issue for succesful quality improvement, this paper is corcerned to SQA activities at project level. Therefore, I'll try to analyze quality management at project level. Project quality management must reflect both organization's quality strategy and particular (customer) quality requirements (Hall [6]). Before discussing a detailed presentation of project quality activities, it is important to offer some definitions about certain basic concepts that shall be used through this paper.

Quality assurance (QA) is the technical implementation of quality management policy for each project. As it is defined by IEEE/ANSI standards QA is "a planned and systematic pattern of all activities to provide adequate confidence that the item or product conforms to established technical requirements"^]. Software quality assurance (SQA) is not a phase. It is an ongoing process to ensure that the quality of the procedures and processes results in a product that fully meets the user's requirements.

Successful SQA needs planning and control. Software Quality Control (SQC) plays a main role in SQA as feedback for quality improvement decisions. Due to this reason many professionals in software community has payed great attention to SQC (even more, many people has concentrated SQA effort only in SQC). In fact, many of the SQA tools are SQC-oriented tools.

2 SQM and SQA in software life cycle

I propose IEEE PI074 standard (IEEE [8]) as a reference document for the description of quality related activities during software life cycle. IEEE PI074 (not yet an approved standard) is a standard that tries to define the processes

(and their constituent activities) that are mandatory for development and maintenance of critical software (it is recommended for non critical software). This standard does not prescribe a specific software life cycle model. So, it can be used to define a common and widely accepted set of quality related activities for software projects.

The standard distinguishes between two kind of quality related processes:

Building Quality into Software 635

Software Quality Management (SQM): planning and administration of the actions necessary to provide adequate confidence in the quality of software products

Software Quality Assurance (SQA) activities to provide adequate confidence that the product conforms to established technical requirements

SQM spreads its activities during the entire life cycle. SQM activities include:

"Planning Software Quality Assurance": identifying software quality

objectives derived using contractual reuirements as well as organizational guidelines and planning management organization, responsibilities, and actions (structure of a SQA plan can be obtained from [7]).

Based on the SQA plan (and quality objectives) SQM must develop quality metrics (and methods for data collection and analysis) as required for the project to monitor the attainment of the quality

goals throughout the life cycle. Quality metrics should be related both to software products and to processes of life cycle. Administration and management of software quality during the life cycle to provide adequate planning, reporting and feedback on the

project's ongoing quality status (reviewing and measuring it against the established milestones and metrics in SQA plan). This activity originates Project Quality Assessments reports. Identification of quality improvement needs by using the results

provided in analysis reports for metric data, found defects and users's view analysis (and other appropiate information).

SQA activities (included in SQA plan) are catalogued in IEEE PI074

standard as part of integral processes. These processes are used to ensure the completion and quality of project functions during the entire life cycle. Integral processes include the following processes:

- V&V. Software Configuration Management (SCM). Documentation development. Training.

Although SCM is treated within the SQA plan (see IEEE [7]) there is a specific plan (SCM plan, refered in IEEE [9]) to deal with this discipline. In this paper, SCM is considered as a separated activity non directly related with SQA tasks.

Thus, SQA activities are those included as part of V&V process,

636 Software Quality Management

including verification tasks as reviews and audits, and validation tasks, as testing. Detailed description of these activities include:

V&V planning: identifying processes and process output

information to be verified and validated. V&V methods include audits, reviews, prototyping, inspection, analysis and demonstration. This planning shall be documented in a Software V&V Plan (SVVP, see IEEE [10]).

Execution of V&V tasks specified in SVVP. Collection and analysis of metric data Testing: planning, test specification development, execution and

results analysis (all these processes are explained in detail in IEEE

All these activities generate different reports about development products and processes that shall be managed by SQM function.

3 Tool support in software development

Since 1979 (one of the first use of the term CASE as referred by Schulmeyer et al. [1]) CASE tools has taken on a increasingly broader meaning for software development. CASE is evolving from computer-aided documentation tools to the expected "intelligent methodology companion" predicted by Carma McClure [13]. CASE means "Computer Aided Software Engineering" and this definition implies that main purpose for a CASE tool is to support software processes. Although the meaning of the acronym CASE seems to be clear, many people restricts CASE tools to structured development techniques automation. As mentioned in Mosley [14], the field of CASE tools do not neccesarily have to be confined to classical automated support of structured analysis and design techniques (like some of the well-known CASE tools: TEAMWORK,

EXCELERATOR, IEW, etc.). CASE tools can support a great variety of software processes and, of course, SQA activities.

Of course, many organizations have considered tools to be a means for improving the consistency, repeatibility and definition of SQA activities. Although tools may offer evident benefits to software development and maintenance, studies on the adoption of tools indicate that they alone do not suffice for activities improvement; proper motivation, training and management within carefully planned implementation programme are also needed. However, as J.Cameron [15] says ("tools can sugar the methods pill"), tools constitute at very least a psychological help.

All these benefits seem imposible if we cannot solve two existing problems with SQA tools:

Building Quality into Software 637

Find a selection method that offers a reasoble degree of confidence in adequate support of software processes Find alternative ways to obtain adequate tools for organization's SQA activities when conventional use of marketed tools do not

offer sufficient support

This paper tries to contribute to the solution of these two problems.

4 SQA tool selection

Among the various CASE selection methods, some authors have already

published contributions to the use of CMM (Capability Maturity Model) as an aid to CASE tool evaluation and selection (e.g. Humphrey [16], Huff [17], Fernandez et al.[18]). As James I.McManus [1] says "it must be realized that

considerable disparity can exist between the producer of software and the capability of tool technology". For example, producer's company may be mature enough to apply tools for its software process but tool technology is not enough evolved. And, of course, also (sadly more frecuent) exists adequate tool

technology, but producer's organization may not be mature enough to use it. This situation often leads to absurd problems, as refered in Rock-Evans [19], when the organization tries to introduce CASE. In words of David Lamb [20] "many such tools suffer from an attempt to impose unverified methods on the

development process. A safer approach is to observe what practitioners do, and find ways to help them do those things more easily". CMM allows to know what practitioners are doing and in which way they can improve their work processes.

CMM-based selection of CASE tools focuses on organization software processes maturity in order to acquire significative information. When an organization assesses its own software processes, two key information are

revealed (Schulmeyre et al.[l]):

adequacy of the processes used daily to produce software

maturity to apply tools to improve those processes

Only when software processes are defined and used consistently, tools can help by supporting them; even more, they may constitute the key for success.

Therefore, process support must be the main criteria for tool selection. CMM approach can be applied to SQA tool selection as an special-purpose

CASE tool. In this paper, I suggest SQA tool selection based on CMM using the CASE/CMM map analysis, a CASE selection method that I have already published [18]. In order to be more precise, I propose SQA/CMM map, a restricted version of original CASE/CMM map.

638 Software Quality Management

5CMM

CMM was developed at the SEI (Software Engineering Institute of the

Carnegie-Mellon University) as a basis for assesing the capability of possible future contractors to the United States Department of Defense (DoD).

LEVELS

Optimizing

LEVEL 4 Maneged

LEVELS

Defined

LEVEL 2 Repeatabl*

LEVEL 1

Initial

Figure 1: CMM levels

CMM divides organization software process maturity into five levels (see figure 1). Greater maturity is associated with higher productivity and quality, and with lower development risks.

* Level 1 ("Initial") represents development characterized by constant crises and carried out using methods that change as they go along

(ad hoc), which is still common. SEI surveys show that 84% of companies fall into this category (Humphrey [16]). * Level 2 ("repeatable"), although still intuitive and dependent on individuals, previously mastered tasks are repeated with regularity

and effective management of the process begins. * Level 3 ("defined") is characterized by an institutionalized and specified software process which is totally independent of the individuals applying it, and there is a Software Engineering Process

Building Quality into Software 639

Group (SEPG) to manage process improvement.

At level 4 ("managed"), the process is measured and controlled in the sense that the relations between activities can be understood quantitatively (e.g. the variations in initial activities can be analysed

to determine their expected impact on later activities). Thus, managers have a solid quantitative basis for planning and decision making. Level 5 ("continous optimization") not only provides for the

management of a given process using automated data collection, but also tackles the change and optimization of the process itself (Paulk et

6 SQA/CMM Map analysis

SQA/CMM Map analysis is a restricted version of CASE/CMM map analysis

created by the author (Fernandez et al.[18]). The objective of this map is the representation of project SQA activities or processes which can be assisted by tools. In order to create it, we will need to know a little more about the internal

structure of CMM levels.

Figure 2: CMM Decomposition

The CMM levels are broken down (see figure 2) into key process areas. Each key area is composed of several key practices which, when they are completed collectively, make it possible to achieve the objectives of the area. Some of the key practices can be selected as key indicators of whether the

640 Software Quality Management

objectives are achieved or not. The practices that are selected as indicators can be formulated as individual questions on the maturity questionnaire (which is the basic maturity evaluation mechanism) as can be seen in Paulk et al.[21].

The individual key areas for passing from a lower to a higher level are (Weber et al.[22]):

Level 2: Software requirements management (RM) Project Planning (include resource estimation) (PP)

Subcontractor Management (SM) Project control and monitoring Software Quality Assurance (conformity with standards, procedures

and requirements) (QA) Software Configuration Management (CM) Level 3: Organization process focus

Organization process definition Integrated Software Mangement Software Product Engineering Training Program

Intergroup coordination Peer reviews Level 4: Software Quality Management

Quantitative Process Management Level 5: Process Change Technology Technology Change Management

Defect Prevention

Each of these process areas (a full description of possible processes and activities during life cycle can be obtained from IEEE [8]) can be supported by tools. A tool can support various functionalities within a key area. The functions of each area are measured by indicators that are composed by a series of characteristics. These are measured using a specific questionnaire about the indicators related to a function.

Although all the areas are important, in this paper, we are only interested in those areas related with SQA activities. These areas are (CM is not included due to reasons mentioned above):

Level 2: Software Quality Assurance

Level 3:

Building Quality into Software 641

Review processes Level 4: Software Quality Management

Quantitative Process Management Level 5: Defect Prevention

It is important to realize that non directly related areas may include several activities related to SQA. For example, activity two of level two area RM indicates that "the software engineering group reviews and agrees to the allocated requirements before they are incorporated into the software efforts; allocated requirements are analyzed to determine whether they are: feasible and appropiate to implement in software, clearly stated, consistent with each other and verifiable"(Weber et al.[22]). But these type of activities of non SQA areas are included by SQA area of level two (activity 5 of SQA area indicates that "SQA group reviews representative samples of deliverable and designated non deliverable software products to ensure compliance with the designated process requirements"). So, we restrict our attention to SQA related areas listed above.

SQA areas (selected from CMM whole set of areas) are the basis to create a SQA tools/CMM map like figure 3 that can be used to evaluate and select SQA tools. In order to decide whether a tool provides support for a function within a key area, each function (Key Practice) will be marked with one of the following options (e.g. see figure 4):

Totally Satisfied (TS): when all the function indicator tests have been passed. Partially Satisfied (PS): when most of the tests are passed.

Not Satisfied: when the PS level is not reached. Not Applicable (NA): when the tool does not provide any functionality.

Selection process cannot be separate from organization process improvement. So, SQA tool selection must follow these steps:

1. Create a list of requirements for SQA tools to be adopted (this list must be inspired on a process improvement plan that indicate what CMM activities must be implemented)

2. For each candidate tool create a SQA tool/CMM map to indicate the support offered for each area.

3. Select those tools that offer best support to selected areas based on

two main criteria:

Defect Prevention Levels T5

Optrntlng PS NS Soft Oirtlity KAin Quantitative Pro. Man. Level 4 TS PS Managed NS Review Processes Level 3 TS TS; Total Supjport PS Defined PS: Panda! SiWort NS Software Qua 1, Ass. NS; No Supp)M Level 2 TS Repea table PS NS

Figure 3: SQA CASE/CMM Map.

p 05 01 Q2 05 (Questions) CM &<

Functionl Functions Functions TSiTotal Support Audit Reviews Planned Doc.Pro&d. PS:Pandal Support Key Area: Review Processes NS:Non ,Supported Functions

Figure 4: Support provided by a tool to the Review Processes key area

Building Quality into Software 643

* Horizontal or "current productivity", looking for tools (or tool kits) that provide support at the maturity level to which organization is currently trying to gain access.

* Vertical or "forward-looking", opting for tools (or tool kits) that offer support to higher maturity levels, to which is to be gained access in the future.

These two criteria are not mutually exclusive and it is possible to opt for mixed policies of tool adoption.

4. Evaluate survivors for other typical factors usually included in

general CASE tool evaluation checklists (e.g. Misra [23], Datapro [24], Rock-Evans [19]).

Anyway, what is being offered is a mean for planning CASE/SQA tools

acquisition on the basis of the organization's prospects for improvement. This approach prevents not desired surprises with respect to lack of coordination between methods used and tools, and the problem of non-forward-looking purchases. For example, CMM approach allows to realize that certain SQA

activities cannot be adressed until a certain maturity level were reached. This information can explain why it is almost impossible to introduce certain SQA techniques in not prepared development environments (when CMM is not

known, it is argued that the reason is "the lack of a proper culture").

7 SQA tools situation

CMM based selection of SQA tools provides an adequate degree of confidence about the adequacy of selected tools respect to development methods. But selection could be useless if available tools are not adequate to organization's

purpose. This problem concerns with the poor availability of SQA tools. As stated in Lamb [20] research have spent a considerable effort on tools restricted to a small portion of the development process. Analysing SQA tools situation, the feeling is that there is no great amount of commercially available tools for

supporting the different activities when we compare its number with other kind of software tools like analysis and design workbenchs that support automation of structured techniques. This situation often leads to SQA professionals to take into account alternative ways (apart from conventional use of marketed tools) to support SQA activities. Such options are presented through several industrial experiences.

I suggest the following clasification of SQA support sources:

* specific SQA tools * use of SQA oriented functionalities of certain CASE tools

644 Software Quality Management

8 Specific SQA tools

There are many types of SQA related tools. Several lists of various types of tools have been published in different references (e.g. P.Powell [25], IEEE [26], Schulmeyer et al.[l]). Analyzing these lists it is possible to conclude that commonest types of SQA tools are those ones mainly oriented to V&V and quality control tasks. A possible list of SQA tools may include (P.Powell [25]):

software design models analysis aids assertion processors

cause-effect graphing aids code auditors file comparators control structure analyzers

cross-reference generators data flow analyzers execution time estimators/analyzers etc...

Unfortunately, a great number of these tools are not commercially available. They have been developed for restricted university or company environments. Besides, it is possible to obtain a more specific support to our particular organization SQA activities by using commercially available tools in non conventional ways. Therefore, apart from the use of marketed specific SQA tools with its evident functionality, it is important to experiment with the following options:

8.1 Use of a specific SQA tool by adapting its functions to organization activities

In a major Spanish telecommunications company, LOGISCOPE is used to provide a mean of carrying out Pareto analysis of code. LOGISCOPE is a SQA tool marketed by Verilog that offers two main functions:

Source code static analysis through code (and certain design) metrics Test generation aids and test execution instrumentation.

This company introduced code inspections (main development language is C) as a SQA activity. As stated in activity 5 of SQA key area in CMM model, due to economic reasons, SQA can only concentrate its effort on code samples for reviewing them. In this company, Logiscope code metrics reports

(with additional Lint reports about code) allow to select parts of code that seems to be defect-prone (B.Kitchenham [4] suggest also the use of Fan-in and Fan-out metrics to detect defect-prone modules, candidates to be inspected).

Besides, Logiscope results format were modified to include relevant data for SQA report administration. Moreover, SQA department creates a networked administration environment to support code inspections burocratic work.

8.2 Construction of a SQA specific tool This has been a common practice in many organizations (e.g. in the same communications company a integrated testing environment for message manage was created using UNIX facilities). The difficulty of tool construction can be cut down by using certain utilities.

For example, FLEXCHECK project at Software Engineering Department of Facultad de Informatica de Madrid has develop a code control tool like Logiscope with improved characteristics. This tool will run on MS-Windows PC environment and will be able to offer total source code language flexibility through the use of LEX/YACC utility (marketed by Abraxas Software). LEX/YACC is a utility that allows the creation of language recognition programs, just providing it a simple grammar definition of language to be recognized. Although this idea is not new (see Tsalidis et al.[29]), it is

FLEXCHECK tool will offer adaptation, virtually, to any procedural programming language by defining its grammar.

9 Using general purpose CASE tools fuctionalities

Another way to obtain support for SQA activities can be derived of the use of traditional CASE tools: analysis and design workbenchs. Two main SQA uses can be derived from these CASE tools:

9.1 Use of SQA oriented functionalities

Most of structured techniques workbenchs offer functionalities that can be used to support certain SQA verification (like review processes) or validation activities (like prototyping). Software Engineering Department of Facultad de Informatica de Madrid have created for SEINCA (a Spanish computing services company for banking institutions) checklists for Software Requirements Specifications (SRS). Checklists were inspired in Yourdon recommendations [27] focused on completeness, integrity, correctness and quality checks. Three different checklists were created three types of SRS:

SRS based on natural language Specifications based on structured analysis EXCELERATOR based specifications.

We could realize that all completeness and integrity checks for EXCELERATOR 1.9 (a CASE workbench marketed by Intersolv) specifications can be solved through EXCELERATOR verification functions (e.g. detection

646 Software Quality Management

of "black-hole" processes can be easily performed through XL Dictionary verification analysis capabilities). Thus, human resources can be employed on non routine activities (i.e. assesing specifications quality).

One more important conclusion about this experience refers to the lack of standardization (definition) of SRS document in this organization. A formal SRS allows to perform more adequate V&V tasks. So, theory of CMM levels is applicable: before introducing certain techniques (reviews), organization must reach adequate maturity level (repeteable/defined level: well- defined deliverables).

9.2 Use of Development Products stored in CASE repository It is possible (depending on the availability of information about internal format of CASE repository) to construct certain programs to extract information about development products, especially for computing certain metrics. For example,

BYL (a Mark II Function Point software), through XL Reconnaissance import module, is capable to extract transaction information from EXCELERATOR repository in order to compute Mkll Function points. Although Mkll FP is not a SQA oriented metric (it is used for cost estimation) it is easy to imagine what kind of metrics can be compute using repository information. For example, tree impurity of modular software design, that is a validated measure introduced by Fenton [28]:

2(e-n+l)/(n-l)(n-2) e: num. of edges, n: num. of nodes

Of course, this specific measure could be compute using information about software structure of modules extracted from a specific SQA tool like LOGISCOPE. In fact, at Software Engineerig Department of Facultad de Informatica de Madrid a program was constructed to compute this metric (and other ones) from LOGISCOPE internal files (the tool can generate the global call graph and control flow diagrams for each module by analyzing source code).

Of course, there are other ways to obtain SQA oriented facilities (e.g. using operating systems facilities) but, normally, they strongly depend on each particular environment.

10 Conclusions

The use of tools to support software processes imply a great potential profit. For this reason a great deal of effort has invested in analyzing best way to incorporate CASE (SQA) tools to software development and maintenance. The optimal introduction of tool support will start with a clear understanding of what SQA activities must be implemented according to organization's maturity level. So this approach leads to maturity self-assesment. Depending on maturity

level, tool support should be planned for a very specific set of SQA activities.

Although organizations often insist on the idea of introducing tools "at any price", CMM indicates that certain techniques cannot be implemented with current maturity level and, therefore, tools for supporting them are useless. As Humphrey [16] says "if your process level is chaotic, control it before

introducing tools".

The availability of adequate SQA tools is often critical. The use of evident functionalities of marketed SQA tools is not always the only way to

provide the neccesary support to specific organization activities. Through the industrial experiences presented we can realize the great possibilities of using, besides SQA specific tools, special functions of traditional CASE workbenchs.

Particularly, this last option is very interesting because we can obtain an additional value to the evident benefits of traditional CASE tools.

References

[1] Schulmeyer, G.,McManus, J.I. (eds.). Handbook of Software Quality Assurance. Van Nostrand Reinhold, New York, USA, 1992.

[2] Fernandez, L., Cruces, J.J. Gestion de calidad del software: ^bueno, bonito,barato? Novdtica, n.102, pp.57-63, March-April 1993, Barcelona, Spain.

[3] Vincent, J.P., Albert, L.W., Sinclair, J.S. Software Quality Assurance. Prentice-Hall, Englewood Cliffs, NJ, USA, 1988. [4] Barbara Kitchenham. "Software Quality, Metrics and Testing. Lecture 3: Cost Estimation for Software Projects", Notes, EuroPACE video course

SE-06, Spring, 1990 (EuroPACE, CNIT-INFOMART, Paris-La Defense, France). [51 ISO. Standard 8402. Quality: Vocabulary, 1986. [6] Hall, P. "Software Quality, Metrics and Testing. Lecture 4: Software

Quality Assurance", Notes, EuroPACE video course SE-06, Spring, 1990 (EuroPACE, CNIT-INFOMART, Paris-La Defense, France). [7] IEEE. IEEE Standard for Software Quality Assurance Plans. IEEE Std. 730-1984. Approved June 14, 1984.

[8] IEEE. Standard for Software Life Cycle Processes. Preliminary standard, subject to revision. P1074/D5. December 1, 1989. [9] IEEE. IEEE Standard for Software Configuration Management Plans".

IEEE Std.828-1983. Approved June 23, 1983. [10] IEEE. ANSI/IEEE Standard for Software Configuration Management Plans. IEEE Std. 1012-1987. Approved February 10, 1987. [11] IEEE. IEEE Standard for Software Test Documentation. IEEE Std.829-

1983. Approved ,1983. [12] IEEE. ANSI/IEEE Standard for Software Configuration Management Plans. IEEE Std. 1008-1987. Approved July 28, 1986. [13] McClure, C. CASE. La automatizacion del software. Ra-Ma, Madrid,

648 Software Quality Management

Spain, 1992. [14] Mosley, V. How to Assess Tools Efficiently and Quantitatively. IEEE Software. May, 1992.

[15] Cameron, J.R.(ed.). JSP and JSD: The Jackson Approach to Software Development". IEEE Computer Society Press, Washington, USA, 1983. [16] Humphrey, W.S. CASE Planning and the Software Process. Journal of

Systems Integration, pp.321-337, 1991. [17] Huff, C.C. Elements of a Realistic CASE Tool Adoption Budget. Communications of the ACM, Vol.35, Num.4, pp.45-54. April 1992. [18] Cervera, J., Fernandez, L., San Feliu, T. "Selection of CASE Tools: New

Approach Based on Capability Maturity Model", pp. 22-31, Proceedings of CIL93. Barcelona, Spain, April 20-22 1993. [19] Rock-Evans, R. Introducing CASE. Ovum Report. Ovum Ltd., London, UK, December 1991.

[20] Lamb, D.A. Software Engineering. Planning for change". Prentice-Hall, Englewood Cliffs, NJ, USA, 1988. [21] Paulk, M.C. et al. Capability Maturity Model for Software. Technical

Report CMU/SEI 91-TR-24. Software Engineering Institute, Pittsburgh, Pennsilvania, USA, August 1991. [22] Charles V.Weber et al. Key practice of the capability maturity model. Technical Report CMU/SEI-91-TR-25. Software Engineering Institute,

Pittsburgh, Pennsilvania, USA, 1991. [23] Misra, S.K. Analysing CASE Systems Characteristics: Evaluative Framework. Information and Software Technology, Vol.32, Num.6, pp.415-422. July/August 1990.

[24] Datapro. How to Evaluate and Select CASE Products. McGraw-Hill. August 1989. [25] Powell, P.B. Software Validation, Verification and Testing Technique and Tool Reference guide. NBS Special Publication 500-93, Washington, 1982.

[26] IEEE. IEEE Standard for Software Quality Assurance Planning. IEEE Std.983-1986. [27] Yourdon, E. Structured Walkthroughs. Yourdon Press, Englewood Cliffs,

NJ, USA, 1989. [28] Fenton, N.E. Software Metrics. Chapman & Hall, London, UK, 1991. [29] Tsalidis, C. et al. ATHENA: A Software Measurement and Metrics Environment. Software Maintenance: Research and Practice, Vol.4,

pp.61-81, 1992.