Inspection's Role in Software Quality Assurance

focusguest editors’ introduction Inspection’s Role in Software Quality Assurance David L. Parnas, University of Limerick Mark Lawford, McMaster University espite more than 30 years’ effort to improve software quality, companies still release programs containing numerous errors. Many major products have thousands of bugs. It’s not for lack of D trying; all major software developers stress software quality assurance and try to remove bugs before release. The problem is the code’s complexity. It’s easy to review code but fail to notice significant errors. Researchers have responded to these problems by studying methods of formal correctness verification for programs. In theory, we now know how to prove programs correct with the same degree of rigor that we apply to mathematical theorems. In reality, this is rarely practical and even more rarely done. Most research papers on verification make simplifying assumptions (for example, a 1:1 correspondence between variables and vari- able names) that aren’t valid for real programs. Proofs of realistic programs involve long, complex expressions and require pa- tience, time, and diligence that developers don’t think they have. (Interestingly enough, they never have time to verify a program before release, but they must take time to re- spond to complaints after release.) Inspection methods can be more effective than informal reviews and require less effort than formal 16 IEEE SOFTWARE Published by the IEEE Computer Society 0740-7459/03/$17.00 © 2003 IEEE proofs, but their success depends on having a I The TSE papers do communicate the re- sound, systematic procedure. Tools that sup- search results. We considered a paper to port this procedure are also important. make a valid contribution if it showed Connecting The Workshop on Inspection in Software how to apply known results to the prob- Engineering (WISE), a satellite event of the lem of inspecting software for suitability theory with 2001 Computer Aided Verification Confer- (fitness for use). The papers are intended practice is the ence (CAV 01), brought together researchers, for people who are willing to read de- essence of any practitioners, and regulators in the hope of tailed, careful research papers. finding new, more effective software inspec- type of tion approaches. Submissions described how We intend that each TSE paper and its corre- engineering. practitioners and researchers were performing sponding Software article have little overlap. inspections, discussed inspections’ relevance, One reports results; the other explains how to provided evidence of how refinement of the use those results and perhaps what research is inspection process and computer-aided tool still needed. support can improve inspections, and ex- We hope that future guest editors will emu- plained how careful software design could late and improve the linked-special-issues ap- make inspections more effective. The best proach for other topics important to both ideas from the workshop have been distilled practitioners and researchers. After all, con- into pairs of articles appearing in linked spe- necting theory with practice is the essence of cial issues of IEEE Software and IEEE Trans- any type of engineering. actions on Software Engineering. What we mean by inspection Why two linked special issues? By inspection we mean a systematic ap- As guest editors, we had a specific goal proach to examining a program in detail. Such when we proposed the joint special issues to an examination’s goal is to assess the quality the publications’ editorial boards. We had ob- of the software in question, not the quality of served that the practitioners habitually neglect the software development process. the kind of research found in TSE on the In general, inspection means examining a (sometimes correct) assumption that it’s irrele- product by following a prescribed, systematic vant to them. On the other hand, researchers process that aims to determine whether the tend to write for each other and to lose con- product is fit for its intended use. For exam- tact with the realities that practitioners must ple, many jurisdictions require vehicle safety face. The linked issues try to narrow this gap. inspections. (Some advocates of specific ap- Some contributors to WISE were practitioners proaches to software inspection assume that who explained what they’re doing and what their method defines “inspection.” In fact, the problems they encounter. Others were re- word was well defined much earlier.) They leg- searchers trying to discover and verify (either islate a list of parts that must be examined, mathematically or experimentally) some fun- measurements that must be made, and so on, damental facts. We thought that these re- and criteria for passing the inspection. The searchers should write articles that explained word “inspection” usually implies that the to practitioners why the problems they were process is described in documents (for exam- studying were relevant to practice. We also ple, checklists and printed forms) that explain thought that the practitioners could communi- exactly what the inspectors must do. These cate to researchers what inspecting a program documents try to ensure that each inspection is actually like. In addition, we asked that the is so careful and so complete that an inspec- purpose of each paper in one publication be tion’s failure to reveal any defects justifies hav- explained to the readers of the other. ing great confidence that the product will per- To summarize: form as required. An inspection process, while systematic I The Software articles aim to make practi- and tightly prescribed, isn’t mechanical; the tioners aware of research ideas that they process description guides the inspectors but might be able to apply. These articles don’t isn’t so prescriptive that a machine could per- communicate the research results as com- form inspections without human involvement. pletely as a normal research paper would. Success depends on the inspectors understand- July/August 2003 IEEE SOFTWARE 17 ing the product and the underlying technolo- cation helps determine mathematical correct- gies, knowing how to use the appropriate ness, but you can have error-free (mathemati- Early inspection tools, and having considerable experience do- cally correct) code that’s hard to understand ing related work. and difficult to maintain. In addition to find- of a document Because the inspectors, like all of us, have ing errors in code and related software docu- that states limits on their ability to handle details, the key ments, inspection can also help determine system to inspection of any complex product is a pol- whether coding style guidelines are followed, icy of divide and conquer—that is, examining comments in the code are relevant and of ap- requirements small parts of the product in isolation, while propriate length, naming conventions are clear can help insure ensuring that and consistent, the code is easy to maintain, and so on. Although these issues are irrelevant that the correct I Nothing is overlooked to theorem provers, model checkers, and au- system is built. I The inspected components’ correctness tomated testing tools, they are crucial to the implies the whole product’s correctness cost of building and maintaining large software products. The inspection’s organization as a set of You don’t need to wait until code is com- discrete steps must assure that each step is plete to reap inspection’s benefits. Early in- simple enough to carry out reliably and that spection of a document that states system re- one step can be carried out without detailed quirements can help insure that the correct knowledge of the others. Inspection can be system is built. In our experience, even when a time consuming. Moreover, no inspection product’s formal verification uses mathemati- process is perfect. Inspectors might take short- cal requirements, they might not accurately cuts, might inadequately understand what capture the designer’s or customer’s intent. In- they are doing, and might identify a product spection of a requirements document helps as- as acceptable when it isn’t. Nonetheless, a sure that the requirements are capturing the well-designed inspection process can find er- right thing. rors that other methods would miss and can engender great trust. Inspection’s future Although many companies now do inspec- Inspection’s benefits tion, they can do better. The Software articles Industry widely employs testing and the re- in this issue provide insights into how soft- search community widely advocates formal ware practitioners can improve their inspec- verification as methods for improving soft- tions’ effectiveness and applicability today. ware quality. Inspection falls somewhere be- The TSE research papers provide the theoreti- tween the two. Formal verification has yet to cal underpinnings of these suggested improve- catch on with software practitioners, while in- ments and offer insights into how we could spection in one form or another has been further improve inspections in the future. The widely adopted by industry and advocated by TSE articles are evidence that inspection con- leading software practitioners (for example, tinues to be an active area of academic research. see Stuart Ball’s Embedded Microprocessor Systems: Real World Design, Newnes, 2000, Refining software inspection pp. 16–24). This difference in acceptance has One way that researchers and practitioners three main causes: are addressing current inspection techniques’ limitations is by refining inspection methods I You can inspect the code itself, not just to make them more appropriate for a particu- abstract models of it. lar setting. Such refinement will help inspec- I Inspection doesn’t require as substantial a tors focus on the most important problems in training investment as verification. the sea of details. I Inspection doesn’t require the time and the In “Practical Code Inspection Techniques for formula manipulation ability that verifica- Object-Oriented Systems: An Experimental tion of typical programs does.

Load more