Smart Grid for Domestic Load Management

NCECT'14 Department of CSE Kalasalingam Institute of Technology A Study on Empirical Evaluation of Mutation Testing for Improving the Test Quality of Safety-Critical Software S.Saranya1, M.Manikandan2, 1-Student, 2-Assistant professor M.E(Software Engineering) Mount Zion College of Engineering Technology Email: [email protected] Abstract— Testing provides a primary means for assuring evaluate their effectiveness. We conducted a major software in safety-critical systems. To demonstrate, experiment to engage 34 programming teams to particularly to a certification authority, that sufficient testing independently develop multiple software versions for an has been performed, it is necessary to achieve the test industry-scale critical flight application, and collected faults coverage levels recommended or mandated by safety detected in these program versions. To evaluate the standards and industry guidelines. Mutation testing provides effectiveness of software testing and software fault an alternative or complementary method of measuring test tolerance, mutants were created by injecting real faults sufficiency, but has not been widely adopted in the safety- occurred in the development stage. The nature, critical industry. In this study, we provide an empirical manifestation, detection, and correlation of these faults were evaluation of the application of mutation testing to airborne carefully investigated. The results show that coverage software systems which have already satisfied the coverage testing is generally an effective mean to detecting software requirements for certification. Specifically, we apply faults, but the effectiveness of testing coverage is not mutation testing to safety-critical software developed using equivalent to that of mutation coverage, which is a more high-integrity subsets ,to identify the most effective truthful indicator of testing quality. We also found that mutant types, and analyze the root causes of failures in test exact faults found among versions are very limited. This cases. Our findings show how mutation testing could be result supports software fault tolerance by design diversity effective where traditional structural coverage analysis and as a creditable approach for software reliability engineering. manual peer review have failed. They also show that several Finally we conducted domain analysis approach for test testing issues have origins beyond the test activity, and this case generation, and concluded that it is a promising suggests improvements to the requirements definition and technique for software testing purpose coding process. Our study also examines the relationship between program characteristics and mutation survival and II. MUTANT CREATION considers how program size can provide a means for RCS was required for source control for each team. Every targeting test areas most likely to have dormant faults. code change of each program file at each check-in can Industry feedback is also provided, particularly on how therefore be identified. Software faults found during each mutation testing can be integrated into a typical verification stage are also identified. These faults were then injected life cycle of airborne software. into the final program versions to create mutants, each contain one programming fault. We elected 21 program Index Terms— Mutation Testing, airborne software, safety- versions for detailed investigation, and created 426 mutants. critical systems. We disqualified the other 13 versions as their developers did not follow the development and coding standards which I. INTRODUCTION were necessary for generating meaningful mutants from Software testing and software fault tolerance are two major their projects. techniques for developing reliable software systems, yet The following rules are applied in the mutant creation limited empirical data are available in the literature to process: 1 NCECT'14 Department of CSE Kalasalingam Institute of Technology 1. Low-grade errors, for example compilation error and core dump exception, are not created. 2. Some changes were only available in middle versions. For example, the changes between 1.1 and 1.2 may not be completely identified in the final version. These changes are then ignored. 3. Code changes for debugging purposes are not included. 4. Modifications of the function prototypes are excluded. 5. As the specification does not mention about memory leaks, mutants are not created for any faults leading to memory leaks 6. The same programming error may span in many blocks of code. For example: a vector missed the division by 1000.0 may occur everywhere in a source file. It is counted as a single fault. III. MUTATION TESTING Mutation Testing is a fault-based testing technique which provides a testing criterion called the “mutation adequacy score”. The mutation adequacy score can be used to measure the effectiveness of a test set in terms of its ability to detect faults. The general principle underlying Mutation Testing work is that the faults used by Mutation Testing represent the mistakes that programmers often make. By carefully choosing the location and type of mutant, we can also simulate any test adequacy criteria. Such faults are deliberately seeded into the original program, by simple syntactic changes, to create a set of faulty programs called mutants, each containing a different syntactic change. To assess the quality of a given test set, these mutants are executed against the input test set. If the result of running a Fig. 2 Process of Data mutant is different from the result of running the original program for any test cases in the input test set, the seeded III. TESTING SAFETY-CRITICAL SOFTWARE fault denoted by the mutant is detected. One outcome of the Mutation Testing process is the mutation score, which indicates the quality of the input test set. The mutation score Test technology is crucial for successful product is the ratio of the number of detected faults over the total development. Inappropriate or late tests, underestimated number of the seeded faults. testing effort, or wrong test technology choices have often led projects to crisis and frustration. This software crisis results from neglecting the imbalance between constructive software engineering and analytic quality assurance. In this article we explain the testing concepts, the testing techniques, and the test technology approach applied to the patient monitors of the HP Omni Care family. Patient monitors are electronic medical devices for observing critically ill patients by monitoring their physiological parameters (ECG, heart rate, blood pressure, respiratory gases, oxygen saturation, and so on) in real time. A monitor can alert medical personnel when a physiological value exceeds preset limits and can report the patient’s status on a variety of external devices such as recorders, printers, and computers, or simply send the data to a network. The monitor maintains a database of the physiological values to show the trends of the patient’s status and enable a variety Fig. 1. Generic Process of Mutation Analysis of calculations of drug dosage or ventilation and 2 NCECT'14 Department of CSE Kalasalingam Institute of Technology hemodynamic parameters. Patient monitors are used in divided into three groups; the Statement analysis group, the hospitals in operating rooms, emergency rooms, and Predicate analysis group and the coincidental correctness intensive care units and can be configured for every patient group. category (adult, pediatric, or neonate). Very often the 2) Mutation Testing for Ada: Ada mutation operators patient attached to a monitor is unconscious and is were first proposed by Bowser in 1988. In 1997, based on sustained by other medical devices such as ventilators, previous work of Bowser’s Ada mutation operators, anesthesia machines, fluid and drug pumps, and so on. Agrawal et al.’s C mutation operators and the design of These life-sustaining devices are interfaced with the patient Fortran 77 mutation operators for MOTHRA , Offutt et al. monitor but not controlled from it. Safety and reliability redesigned mutation operators for Ada programs to produce requirements for medical devices are set very high by a proposed set of 65 Ada mutation operators. According to industry and regulatory authorities. There is a variety of the semantics of Ada, this set of Ada mutation operators is international and national standards setting the rules for the divided into five groups: Operand Replacement Operators development, marketing, and use of medical devices. group, Statement Operators group, Expression Operators The legal requirements for electronic medical devices are, group, Coverage Operators group and Tasking Operators as far as these concern safety, comparable to those for group. nuclear plants and aircraft. In the past, the safety requirements covered mainly the hardware aspects of a device, such as electromagnetic compatibility, radio interference, electronic parts failure, and so on. The concern for software safety, accentuated by some widely known software failures leading to patient injury or death, is increasing in the industry and the regulatory bodies. This concern is addressed in many new standards or directives such as the Medical Device Directive of the European Union or the U.S. Food and Drug Administration. These legal requirements go beyond a simple validation of the product; they require the manufacturer to provide all Fig 3.Publications of the Applications of Mutation evidence of good engineering practices during

Load more