Conception formelle
Frédéric Herbreteau
Slides by Igor Walukiewicz, thanks! Factorial(x) := if x=0 then 1 else x*Factorial(x-1)
Is this program correct? Some examples where it matters Therac-25 Radiation Overdosing (1985-87)
• Radiation machine for treatement of cancer patients • At least 6 cases of overdoses in period 1985-1987 • Three death cases • Source: Design error in the control software (race condition) AT&T Telephone Network Outage (1990)
• 9 hours otage of large parts of US telephone network • Cost: several 100 million US$ • Source: software faw (wrong interpretation of break statement in C) Ariane 5 Crash (1996)
• Crash of Ariane 5 missle in June 1996 • Cost: more than 500 milion US$ • Source: software faw • A data conversion from 64-bit foating to 16-bit signed integer Pentium FDIV Bug (1994)
• FDIV= foating point division unit • 1 in 9 bilion foating point dividers would produce inacurate results • Cost: 500 million in replaced processors • Source faw in a division table BOEING 737-MAX (2019)
• 356 morts (2 crashes) • coût en dizaines de milliards de dollars • problème de design du “fight control software (MCAS)” (et autre) Why it is diffcult to verify computer systems?
•Analog systems are continuous •Digital systems are not •Big number of components interacting together Some analysis
“The role of software in recent Aerospace Accidents”
Nancy G. Leveson Aeronautic and Astronautic Department MIT Accidents analysed •Explosion of Ariane 5 •Loss of Mars Climate Orbiter •Destruction of Mars Polar Lander •Placing Milstar satellite in an incorrect orbit •American Airlines B-757 crash into a mountain near Cali •Collision of Lufthansa A320 into earth bank in Warsaw •Crash of China Airlines A320 near Nagoya •Overconfdence on digital •Flawed review process automation •Revision for undesired •Not understanding the risks behaviours is very productive associated to software •Inadequate safety engineering •Almost all errors were due to faws in specifcation and not in •50%-70% safety decisions are coding made in early stages of development •Reliability techniques (like redundancy) not effective for •Software reuse without safety software analysis
•Assuming the risk decreases •Unnecessary complexity of over time software
•Inadequate specifcations Formal verifcation at work The transputer project
• T9000 transputer (still widely used: GPS) • Formal development of foating- point unit • Formalisation revealed problems with the standard • Diagnostic information should be propagated, but sometimes it was impossible. • Verifcation with Hoare logic revealed errors (on very small class of vectors) • Formal development turned out to be faster • Queen’s Award for Technology Achievement in 1990 Signalling system for RER
• Increase traffc by 25% preserving safety levels • 21K of Modula-2 code have been formally specifed and verifed using B method • Later the same method has been used for line 14 and Roissy Shuttle • No unit test where preformed, just some global tests B method AAMP Microprocessor
• AAMP5 widely used processor (Rockwell Collins) • .5 M transistors • Completely verifed in PVS (300 hours per instruction) • Later verifed AAMP-FV showing dramatic reduction in verifcation costs • National Security Agency certifcation for use in cryptographic applications. Airbus
• Development of Level A controllers for A340/600 series (Esterel technologies) • 70% of code generated automatically • Quick management of requirements changes • Major productivity improvement (each new project requires twice as much software as its predecessor) • SCADE has been adopted for A380 for most of on-board computers. What are formal methods Specifcation + Analysis of the system Customer needs
Requirements (safety,…)
Specifcation
Source code
Machine code
Specifcation
Giving precise statement of what the system must do while avoiding constraints on how it is achieved
Ex: Everybody should be happy
Many observers have noticed that the process of developing a formal specifcation is often as effective for fnding errors as the verifcation effort in which the specifcation is to be used. Analysing system properties
Formal methods allow to reason about systems
Formal specifcation is a mathematical object. One can analyse it, manipulate it, reason about it
A model of the system that suppresses implementation detail allows the architect to concentrate on the analyses and decisions that are most crucial.
Problem: formal specifcations are typically large (thousands of lines) Verifcation is impossible (algorithmically) Halting problem for Turing machines
Alain Turing (1912-1954) Mathematician, Logician, crypto-specialist Computational model: Turing machine
Program termination is not decidable: There is no algorithm to decide if a TM stops on the empty input Methods of system verifcation Peer reviewing
• Manual code inspection. • On average 60% of errors caught. • subtle errors (concurrency, algorithm defects) hard to catch. • Used in 80% of all software engineering projects. • Refnement of this method: parallel developpement Testing 30%-50% of development cost.
Programmers have to provide insights what to test, and what should be system response.
When to stop testing?
Formal specifcations help here:
One of the most cost-effective uses of specifcations
New tools provide as good coverage as manual test cases. They avoid programming test cases Get them as soon as you can Theorem proving
Doing large proofs semi-automatically Model Checking Model checking fow-graph
Requiremen The ACM Turing Award in 2007 for model-checking
Some Turing Award Winners •Edsger Dijkstra (1972) •Donald Knuth (1974) •Michael Rabin and Dana Scott (1976) •Tony Hoare (1980) •Thompson & Ritchie (1983) •Hopcroft & Trajan (1986) •Rivest, Shamir, Adleman (2002) The ACM Turing Award in 2007
Edmund M. Clarke Jr. (CMU USA) Allen E. Emerson (U. of Texas at Austin, USA) Joseph Sifakis (IMAG, Grenoble)
Jury justifcation: For their roles in developing Model-Checking into a highly effective verifcation technology, widely adopted in the hardware and software industries Advantages
Makes formal techniques available to broad audience
Automatic procedure taking on an input: • a fnite state model • and a set of required properties A transition system Properties
The program terminates The program does not terminate The computed value is correct A sequence of events is correct Mutual exclusion No deadlock No starvation No starvation
Model Checker
Yes No Insuffcient memory
Counterexample Advantages •Widely applicable •Allows for partial verifcation •Automatic •Produces counterexamples •Sound and interesting mathematical foundations
Disadvantages •Analyses an abstraction of the system •Focused on control an not data oriented applications •No guarantee about the completeness of the results •Impossible to generalise Milestones towards MC
Mathematical program correctness: Turing, Church (1949)
Syntax based technique for sequential programs Floyd Hoare (69)
Dikstra, Hoare: invariant method and program refnement (70ties)
Program logics •Algorithmic and dynamic logics •Hoare logics •Linear temporal logic
Model checking using temporal logics Clarke Emersion 1981, Sifakis at the same time First appearance of MEC (Alaiwan, Arnold 1983) Who uses formal methods?
• Microsoft Azure (cloud computing) • Amazon S3 (cloud computing) • Intel (processors) • Airbus (aircrafts & space) • Dassault (aircrafts, systems 3DS) • ConsenSys (blockchain, smart contracts) • Oracle (“Parfait” static analysis) • Framatome (nuclear reactors) • Bosch (automotive) • … Any verifcation using model-based techniques is only as good as the model of the system Conclusions
Formal methods are not silver bullet to eliminate errors
They are not beyond budget of system developers
They are the only practical means to demonstrate the absence of undesired behaviour
The process of developing a specifcation is often the most valuable phase
Leading hardware developers continue to apply model chekcing and proof technology In spite of their successes, verifcation technology and formal methods have not seen widespread adoption as a routine part of systems development practice, except, arguably, in the development of critical systems in certain domains. Indeed, we expect diffusion of rigorous verifcation and design technology to take place gradually, and not result in their explicit adoption as a distinct technology
Hoare's vision: computer software as a most reliable component in any system it controls, and no one can blame the software anymore Plan of the course 1. Test vs. Model-checking vs. Proof 2. Formal modelling 3. Linear Temporal Logic (LTL) 4. LTL model-checking algorithms 5. Verifcation of algorithms with TLA+ 6. Fairness 7. Incremental modelling 8. Verifcation of infnite state algorithms