ON THE DESIGN OF SIDESTICK CONTROLLERS IN FLYBYWIRE

AIRCRAFT

Muy Thomas and Bowen Ormsby

University of Glasgow Scotland UK

resp ect to some safety requirements Two mo dels of increas Abstract

ing complexity are develop ed and analysed using the ISO

formal description technique LOTOS Language of Tempo

This pap er presents the problem of designing the functional

ral Ordering Specication LOTOS is appropriate since

b ehaviour of the interaction b etween two sidestick con

we are concerned only with relative time rather than with

trollers an autopilot and a ightcontrol computer in a

real time

ybywire aircraft Two mo dels are develop ed using the

The rst section gives the informal requirements of the

ISO formal description technique LOTOS and analysed us

sidesticks controllers and the second section intro duces LO ing rigorous abstract testing techniques

TOS The basic design description is given in the third sec

Keywords Avionics Design Formal Description Tech

tion followed by an analysis of the design Then the design

niques LOTOS Pro cess Algebras SafetyCritical Software

is extended to include time and this to o is analysed There

Sp ecication Testing Verication Validation

follows a discussion and our conclusions

Intro duction

Requirements

The reliabili tyofsoftwareinemb edded computer systems is

The requirements have b een extracted from

a matter of increasing concern particularl y for safetycritical

Each pilot has a sidestick and the two sticks are linked

systems

to the ightcontrol computer Normally b oth sticks are

There are no known quantitative metho ds for demon

enabled and movements of them pro duce a resp onse which

strating high levels of software reliabili ty instead

dep ends on the sum of their displacements up to a maximum

most approaches to software in safetycritical situations rely

of the full deection of one stick So if b oth sticks are moved

mainly on assurance based on the evidence pro duced dur

forward two degrees the aircraft will b ehave as if one stick

ing the software lifecycle Our interest lies in augmenting

had b een moved forward four degrees Similarly equal and

the preliminary and detailed design stages with mathemati

opp osite movements cancel each other out

cal mo delling and validation stages our goal is the reduction

In an emergency one pilot can override the other pilots

of errors and design changes during unit testing after imple

controls by pressing a button on hisher stick This button

mentation ie by nding the errors before implementation

which also acts as the autopilot disengage button causes

us we are interested in applying formal metho ds as design Th

inputs from the other pilots stick to b e ignored and gives

to ols

prioritycontrol to the pilot who pressed it If the button is

The aerospace industry is an area where software on

held down for more than thirty seconds the stick retains pri

the whole is extending existing electromechanical systems

orityuntil the system is reset bycentralising the disengaged

the concerns of control and protection must b e clearly sep

stick This override system allows one pilot to take sole con

arated and qualied with a high degree of assurance It

trol of the aircraft if the other pilot gets into dicultyorif

is b ecoming clear from our own exp erience from others in

a stick jams

the eld and in the aerospace industry eg

Amongst many other controls b oth pilots also havean

that techniques such as static analysis and validation based

autopilot engage button If b oth pilots are in control ie

on mathematical mo dels can have an imp ortantroletoplay

not overridden and one of them presses this button the au

in the design of safetycritical software This is particularly

topilot also gains control of the aircraft until one of the pilots

the case when analysis addresses the questions of whether

presses the autopilot disengage button on hisher sidestick

the software satises safety constraints

A pilot in control can engage or disengage the autopilot but

Here we consider a case study the problem of design

a pilot not in control cannot aect the state of the autopilot

ing the functional b ehaviour of the interactions b etween two

sidestick controllers the autopilot and a ightcontrol com

Design Descriptions

puter in a ybywire aircraft This is a relevant example

since such a conguration is part of the current Airbus Indus

The natural language description given ab ove whilst rela

trie A Our aim is to develop a mo del from an initial

tively clear is still ambiguous and incomplete Further it

statement of the requirements to verify that the mo del do es

oers no p ossibili ty of rigorously demonstrating that it is a

full the requirements and to provide some degree of assur

go o d safe proto col that can b e implemented

ance that this is a go o d design by analysing the mo del with

Our intention is to develop a design description whichis

closer to an eventual implementation but abstract enough Funded by University of Glasgow Research Studentship

opns nnnnpppp stickpos to allow for rigorous analysis of the essential b ehaviour

Succ Pred stickpos stickpos The formal description technique LOTOS is used to

eq mo del the design LOTOS is a pro cess algebra similar to stickposstickpos stickpos

CCS but with data typ es and multiway synchronisa eqns forall xystickpos

tion which allows us to mo del pro cesses that can b e non

deterministic andor concurrent with or without synchro endtype

nisation Pro cesses can b e parameterised by abstract data

A pilots controls consists of the x and y co ordinates of

typ e values and suchvalues can b e passed or negotiated

a stick the co ordinates are selected with op erations xstick

between pro cesses through synchronisation

and ystick and the co ordinates are incremented and decre

A good intro duction to LOTOS is given in a very

mented with op erations xinc xdec etc The op eration

brief review of the features of LOTOS we use is as follows

central is a predicate to test whether b oth co ordinates are

Pro cesses are built up from constant pro cesses events and

at the origin Again many of the equations are omitted

pro cess op erators Events are atomic indivisi bl e actions In

the following P and Q are pro cesses

type pilotcontrols is stickpos

sorts pcontrols

LOTOS Informal description

opns mk stickposstickpos pcontrols

exit termination

xstickystick pcontrols stickpos

aP prex P byevent a

xincxdec pcontrols pcontrols

PQ choice b etween P and Q

yincydec pcontrols pcontrols

PQ b ecome Q after P terminates

central pcontrols bool

P Q P in parallel with Q

eqns forall xystickpos

P l Q P in parallel with Qsynchronisin g

ofsort stickpos

on events in list l

xstickmkxy x

exp P if exp holds then b ecome P

ystickmkxy y

ofsort pcontrols

Abstract data typ es are sp ecied in the ACT ONE sub

xincmkxy mksuccxy

language using manysorted equational logic Values and

xdecmkxy mkpredxy

pro cesses are combined in twoways lists of values maybe

asso ciated with events and pro cesses may b e parameterised

ofsort bool

byvalues We use two forms of value asso ciation with events

centralmkxy x eq and y eq

the event oer av oers value v at event a and the event

endtype

oer avT oers anyvalue of typ e T at event aTwoevents

oering equivalentvalues may synchnronise eg atrue can

There are distinct values for priority status

synchronise with atrueor anotfalse and an event of

fer av can synchronise with an event oer avT with

type priority is boolean

the eect that v is b ound to v when v has typ e T

sorts priority

The underlying semantics of a LOTOS description is a

opns onetwonone priority

state transition system rather like a nite state automata

eq priority priority bool

except that there is no restriction to nite states We use

eqns

laws which describ e equivalences b etween pro cesses based

on the concept of weak bisimulati on b etween transition

endtype

systems Bisimulati on is socalled b ecause the pro cesses can

simulate each other These laws include for example the

There are values for autopilot status

rule that choice is commutative and that parallel pro cesses

type autopilot is boolean renamedby

expand into pro cesses involvin g only choice and prex

sortnames autopilot for bool

For brevity all the events in our descriptions will b e ob

opnnames on for true

servable This will allow us to omit the usual list of ob

off for false

servable parameter events in pro cess declarations except

endtype

in the few cases where the actual and formal events do in

deed dier We also intro duce some further trivial abuses

The main pro cess comp onents of the system are the two

of LOTOS notation for brevity

human pilots referred to as pilot and pilot the autopi

lot and the ightcontrol computer

Basic Design

The twohuman pilots b ehaviours are identical up to

the names of the events and so the pro cess is parameterised

We b egin with a design of the basic system excluding timing

Each pilot maymove the stick along one of the four axes

constraints

forward Fd backward Bk left Lt rightRt depress or

Four data typ es are dened to represent the status of

release their priority button PandPr resp ectivelyand

a stick a pilots controls pcontrols the control priority

depress or release their autopilot button AUandAuD

priority and the autopilotIntheinterest of brevity

resp ectively

most of the obvious equations are omitted from these

typ es

process PilotFdBkLtRtAuAuDPPR

A stick can b e in any p osition along an axis from n neg

FdBkLtRt

ative toppositive with as origin Stick p ositions

Pr Au P AuDPilot

may b e summed with over and underows rounded up or

exit

down to n p resp ectively

endproc

type stickpos is boolean The autopilot has a more restricted functionali tywhich

sorts stickpos includes only stickmovements

where process AutoPilotFdBkLtRt

Pilot PilotFBLRAADPPR Fd Bk Lt RtAutoPilot

Pilot PilotFBLRAADPPR and exit

AutoPilot AutoPilotFBLR endproc

The ight control computer p olls for the current

The State pro cess is where the b ehaviour of the interaction

summed p ositions of the sticks and the priorities with event

between the pilots is dened All pilot events are p ossible

s and sends on the appropriate signal with event signal

ie a pilot can always apply a force to hisher stickbutton

to the control surfaces There are several p ossibiliti es for

but that eventmayhave no observable eect Since much

combining the inputs into the appropriate signal dep ending

of the details concerning the three pilots is similar wehave

on whichhuman pilot if any is in overall control and

omitted parts of this pro cess unfortunately LOTOS is not

whether or not the autopilot is engaged

higher order and do es not allow parameterisation over pro

cesses

process FCC

spipipipcontrolsoneon

s process Statepipipipcontrol

signalmkxstickpixstickpi

ppriorityaautopilot

ystickpiystickpi

FStateyincpipipipa

FCCssignal

FStatepiyincpipipa

spipipipcontrolsoneoff

FStatepipiyincpipa

signalmkxstickpiystickpi

FCCssignal

similarly for BBBRRRLLL

ntrolstwoon spipipipco

signalmkxstickpixstickpi

Pnot p eq twoStatepipipioneoff

ystickpiystickpi

p eq two Statepipipipa

FCCssignal

Pnot p eq oneStatepipipitwooff

spipipipcontrolstwooff

p eq one Statepipipipa

signalmkxstickpiystickpi

PRp eq one Statepipipinoneoff

FCCssignal

notp eq oneStatepipipipa

spipipipcontrolsnoneon

PRp eq two Statepipipinoneoff

signalmkxstickpixstickpixstickpi

notp eq twoStatepipipipa

ystickpiystickpiystickpi

Anotp eq two Statepipipipon

FCCssignal

p eq two Statepipipipa

ff spipipipcontrolsnoneo

Anotp eq one Statepipipipon

signalmkxstickpixstickpi

p eq one Statepipipipa

ystickpiystickpi

ADnotp eq twoStatepipipipoff

FCCssignal

p eq two Statepipipipa

exit

ADnotp eq oneStatepipipipoff

endproc

p eq one Statepipipipa

spipipaStatepipipip a

The eects of the pilots actions on the FCC are dened

exit

by the pro cess State which is comp osed in a constraint

endproc

oriented style with the three pilots and the FCC The

constraintoriented style is one in which constraints are

Note that the eect of the priority and autopilot button

regarded as b ehavioural prop erties of the system and the

events dep end on the overall pilot priority

complete system is formed bycombining all of these prop er

ties using a form of parallelism Unconstrained parallelis m

Analysis

corresp onds to the disjunction of the b ehavioural prop er

ties and constrained parallelism ie with synchronisatio n

Our analysis of the design has two aims to verify that it

corresp onds to a form of conjunction dep ending on the syn

do es fulll the requirements and that to showthatitisa

chronising events

go o d design with resp ect to some basic safety prop erties

The constraintoriented style is the b est approach for

Some of them are

this problem for two reasons First in the design pro cess it

allows us to separate concerns and decouple the main com

Moving a pilots stick cannot aect the state of control

p onents of the system the pilots and the FCC We can con

nor of the autopilot

sider them in isolation as indep endent agents and then de

If the autopilot is on then the signal sums the au

ne their interaction Second this approach b est reects the

topilot stick p ositions with the pilots stick p ositions

separation of comp onents in the intended hardwaresoftware

dep ending on who is in control

implementation

If the autopilot is o then the autopilot stick p ositions

The overall pro cess is given by the appropriate combina

are not summed in the signal

tion of the ve comp onents

If b oth pilots are in control ie neither is overridden

process Controllerpi pipipcontrols

and one of them presses the autopilot button the au

ppriorityaautopilot

topilot also gains control of the aircraft until one of the

Statepipipipa

pilots presses the autopilot disengage button on their

all events

sidestick

Pilot Pilot AutoPilot

s A pilot in control can engage or disengage the autopi

FCCssignals lot but a pilot not in control cannot aect the state of

endproc the autopilot

Demonstrating that the other prop erties hold involves Unlike the description of the ightwarning computer in

considerably more complex test pro cesses It is easier to LOTOS in we are able to p erform rigorous analysis di

demonstrate that a renement of a prop erty holds For ex rectly on our design In blackb ox testing was p er

ample we can easily show a renement of prop erty formed on an ADA implementation

Sp ecicallywe p erform an abstract form of testing

Controllerxynoneo ff

called prop erty testing which is describ ed in and used exten

all events

sively in Testing in LOTOS is a form of state reach

P

ability analysis prop erty testing is a more abstract form of

SticksFBRL SticksFBRL

testing for a sp ecic prop erty The prop ertyisdenedas

SticksFBRL

a LOTOS pro cess and then the test pro cess is combined

in parallel synchronisin g on the events of the test pro cess

A spipipipcontrolsnoneoffexit

with the given pro cess Essentiallywe are testing to see if

the given pro cess can b ehave like the test pro cess without

Informallywe are testing for givennopilotisinoverall

deadlo ck If the test is passed then we can b e assured that

control and the autopilot is o if pilot depresses hisher

the prop erty do es hold in that the test b ehaviour is p ossi

priority button and then after anynumber of events which

ble On the other hand if the test is not passed then we

do not include any button events pilot depresses hisher

can conclude only that the b ehaviour is not p ossible for the

autopilot the autopilot is still o

states considered so far

One requirementwhichwe are not able to express and

This illustrates an imp ortant p oint there are always

verify in this design is one concerning time namelythe

more states to explore b ecause there are innitely many

behaviour when a priority button has b een engaged for a

pro cesses ie the pro cesses have the form Pexit

length of time In order to express this we will need to

PWe can only reason conclusively using testing over

extend the design to include a notion of time we will do so

a nite numb er of states However if we detect duplicate

in Section

states then we can reason ab out recursive pro cesses using

a xed p oint theorem This means that if wecannda

Discussion the Design Pro cess recursive denition of the combined pro cess under insp ec

tion then if the test cannot b e passed in the nonrecursive

The LOTOS mo del has b een invaluable it has allowed us to

prexes we can conclude that the test can never b e passed

precisely dene our design to test it and analyse it More

erication Prop erty testing is not the most sophisticated v

over the activity of describing the design in LOTOS made

technique in comparison to say a temp oralmo dal logic

us think very carefully and hard ab out the interactions of

But it is particularl y attractive to us b ecause it can b e p er

the various comp onents of the system Wewere often forced

formed with the software to ol LOLA see a simula

to determine the interaction b etween pilot control and the

tion to ol for symb olic computation that can recognise dupli

autopilot in more detail than was explicitl y discussed in

cate states Also and no less imp ortant we use LOLA for

the requirements For example when the autopilot is dis

straightforward prototyping or animation when developing

y We conclude that it engaged what is the control priorit

the descriptions

remains unchanged

We cannot discuss each prop erty in detail here but as one

Also we did get some asp ects of the design clearly wrong

example consider testing for the rst prop erty Because this

For example at rst we could not show the fth prop erty

is a general prop erty ie not one ab out a sp ecic instance of

This was b ecause in the pro cess State we had the choices

the controller the test pro cess dep ends on the free variables

in the given pro cess say ControllerxypaAt rst it

seems then that the corresp onding test pro cess is the pro cess

ADStatepipipoff ADStatepipipoff

which oers anynumberofstickevents from the three pilots

ie any pilot can switch o the autopilot But the require

followed by an oer of the s eventtheeventwhich passes

ments clearly state that a pilot not in control cannot aect

information to the ight controller with the priority and

the state of the autopilot and in our design a pilot is only

autopilot values unchanged ie

not in control when the other pilot has sole control eg pi

Controllerxypa

lot is not in control when the priorityistwo Fortunately

all events

through the pro cess of testing we found the error and cor

SticksFBRL SticksFBRL

rected it accordingly A nal lesson learned was catchde

SticksFBRL

sign errors earlydevelop the design iterativelyandsavethe

spipipipcontro lspaexit

iterations carefully In other words careful managementof

the design pro cess is essential

where Sticks is just identical to AutoPilot

Using LOLA the test is passed with a suitable depth of

state exploration However this only shows that it is possible

Timed Design

that the stick actions do not aect the priority or autopilot

status it do es not show that they cannot have such an eect

In this section we discuss how to augment our design with an

In order to show this we need to show that tests suchas

explicit mo del of time We will not deal with real time since

we do not know what these constraints are for this problem

Controllerxypon

There are various ways to mo del time wehavechosen to

all events

take an approachwhich is consistentwiththeinterleaving

SticksFBRL SticksFBRL

mo del of paralleli sm rather than true concurrency whic h

SticksFBRL

underlies LOTOS All pro cesses synchronise on one clo ck

spipipipcontrolspoffexit

with the tickevent t The tick is assumed to b e of sucha

cannot b e passed Using LOLA we are able to transform short duration that at most one event can o ccur at each tick

this particular pro cess into a rather large and complicated although no event is required to o ccur

set of recursion equations examination of the nonrecursive First we extend the current description in a constraint

prexes reveals that this test cannot b e passed oriented style with a clo ck pro cess In addition to the tick

Again the pro cess of making the LOTOS description has event this pro cess oers an event now whichisassociated

forced us to consider just the kinds of details not covered by with the currentvalue of the clo ck

the requirements Recall that priority cannot b e given to

the pilot without control unless the disengaged stickhas

process ClocktnowcurrentNat

b een centralised But for example if a pilot has absolute

t ClocktnowSucccurrent

control and the other pilot depresses hisher priority button

nowcurrent Clocktnowcurrent

when hisher stickiscentralised then do es the other pilot

exit

immediately gain control without the rst pilot releasing

endproc

hisher priority button We conclude that the answer is

yes

Each high level pro cess in the controller pro cess is aug

Moreover what happ ens when a pilot who has absolute

mented with an initial oer of the tickevent So for example

control releases hisher priority button We conclude that

instead of Pilot Pilot AutoPilot wedene

this should have no eect on the overall priority when the

a pro cess Pilots

other pilots stick is not centralised The absolute priorityis

usually only relinquis hed after actions from b oth pilots as

process Pilots

suming that the pilot not in control has to centralise hisher

t Pilot Pilot AutoPilot

stick This reects we think the idea that the timing con

Pilots

straint is there to alleviate the need for a pilot to depress the

endproc

priority button for long p erio ds of time and so after a p erio d

Second we extend the design to mo del the b ehaviour

of time depression or release of that button in the absence

when a priority button has b een engaged for a length of

of anycentralising movements to the other stick eg the

time eg seconds We will assume here without loss of

other pilot has had a heart attack conveys no information

generality that a tick represents a second Again we extend

However an overidden pilot can gain control from one in ab

the description in a constraintoriented style with a pro cess

solute control if after seconds heshe centralises hisher

Button which stores the last time of depression of a priority

stick and presses hisher priority button

button Event wb mo dels the write action and event rb the

The State pro cess is where the mo de is determined If

read action

one pilot has sole control then this involves determining

whether or not that pilot has absolute control This is done

process ButtonrbwbtimeNat

by comparing the current time with the last time a priority

wbbtimeNat Buttonrbwbbt ime

button was engaged

rbtime Buttonrbwbtime

process State

exit

pipipipcontrolsppriorityaautopilot

endproc

p eq none Statepipipipa

notp eq none

The pro cess State is extended to reect three dierent

nowtimeNat rbptNat

mo des These are as b efore ie neither pilot has overall

p eq one and time pt gt

control pilot has what we call absolute control or pilot

Stateone

has absolute control By absolute control wemeanthata

p eq one and time pt lt

pilot has had sole priority for more than seconds

Statepipipipa

These three mo des are mo delled by subpro cesses State

p eq two and time pt gt

which is essentially the old pro cess State but it recur

Statetwo

sively calls the new State instead of State Stateone

p eq two and time pt lt

and Statetwo

Statepipipipa

The two latter pro cesses are identical mo dulo event and

endproc

value renaming Stateone for example denes the b e

haviour when pilot has absolute control Since the eects

Finally the synchronisation lists in the overall controller pro

of the stickmovements are exactly as in State these are

cess are amended

omitted

process Controller

process Stateone

ols pipipipcontr

pipipipcontrolsppriorityaautopilot

ppriorityaautopilottimeptNat

t

Statepipipipa

all events

PStatepipipipa

Pilots

nowtimeNatwbtime Pcentralpi

ts

Statepipipitwoa

FCCssignalst

notcentralpiStateonepipipipa

tnow

PRcentralpiStatepipipinonea

Clocktnowtime

notcentralpiStateonepipipipa

rbwb

PRStateonepipipipa

Buttonrbwbpt

AStateonepipipipon

endproc

AStateonepipipipa

f ADStateonepipipipof

ADStateonepipipipa

Analysis

spipipipaStateonepipipipa

This design was considerably more dicult to develop and Stateonepipipipa

analyse than the basic design The use of an automated test exit

ing and simulation to ol suchasLOLAwas crucial as timing endproc

T Bolognesi and E Brinksma Intro duction to the ISO constraints are notoriously dicult to get right and this

Sp ecication Language LOTOS In PHJ van Eijk example was no exception Moreover with the additional

CA Vissers and M Diaz editors The Formal De events the state space has grown enormously and some de

scription Technique LOTOS pages Elsevier Sci gree of automation is essential

ence Publishers BV NorthHolland

The prop erties to consider remain as b efore with the ad

dition of a variety of prop erties ab out the b ehaviour when

Hub ert Garavel and RenePierre Hautb ois Experiment

a pilot gains and loses absolute control Again wedonot

ing LOTOS in Aerospace Industrychapter Amast

give full details here but as an example we describ e how

Series in Computing World Scientic To app ear

we can construct the test if pilot is in priority for more

Flybywire controls The new airliner standard Inter

than seconds pilot s stick is not centralised and pilot

national Civil Aviation Authority Bul letin pages

depresses hisher priority button then pilot is still in

March

priorityWe dene a pro cess which expands into a choice

between instances of the form

International Organisation for Standardisatio n Infor

n m

t P t F t P spipipionea

mation Processing Systems Open Systems Intercon

where n m

nection LOTOS A Formal Description Technique

We consider only one stickevent after the P and it do esnt

Based on the Temporal Ordering of Observational Be

matter which one it is b ecause wehavealreadyshown that

haviour

stickevents cannot aect the status of the priority or au

C Kirkwo o d and M Thomas Exp eriences with LO

topilot Also we do not constrain the o ccurrences of button

TOS Verication A Rep ort on Two Case Studies

read or writes in the test therefore it is imp ortant to exclude

Submitted for publication

rb and wb from the synchronisatio n list b etween the events

B Littlewo o d The Need for Evidence from Disparate

the test and the controller

Sources to Evaluate Software Safety In T Anderson

F Redmill editor Directions in SafetyCritical Sys

Discussion

tems Safety Critical Systems Club

Issues in the developmentofsafety J McDermid

Until recently LOTOS has b een used mainly for descrip

critical systems In T Anderson F Redmill editor

tions in the OSI op en systems interconnection context and

Safety Critical Systems Current issues techniques and

the telecommunications eld some other applicatio ns are

standards Safety Critical Systems Club Chapman and

rep orted in At rst this seemed a new application

Hall

area for LOTOS but in hindsight this problem has many

R Milner A Calculus of Communicating Systems

characteristics of a proto col and LOTOS has b een ideal for

LNCS SpringerVerlag

articulating many of the design decisions It has allowed

indeed forced us to think carefully ab out the interaction

J Rushby Formal Sp ecication and Verication for

between pilots and in particular how and when priorityis

Critical Systems Tools Achievements and Prosp ects

gained and relinquish ed This study is a preliminary inves

Technical Rep ort EPRI TR Electric Power Re

tigation of the problem and future work will consider other

search Institute January

p ossible proto cols for this system

J Rushby Formal Metho ds and the Certication of

Critical Systems Technical Rep ort CSL SRI In

ternational Decemb er

Conclusions

M Thomas The Story of the Therac in LOTOS

LOTOS has allowed us to develop a constraintoriented de

High Integrity Systems Journal

sign description for this problem This approach invites us

K J Turner editor Using Formal Description Tech

to develop and test the imp ortant details of the comp onents

niques An Introduction to ESTELLE LOTOS and

and then to consider their interactions The resulting design

SDL Communication and Distributed Systems Wiley

is one which is close enough to the implementation design

yet abstract enough to allow for rigorous reasoning using

automated to ols We are able to conclude with some con

dence that wehave a go o d safe design

Acknowledgements

We thank Tom Melham for his comments

Addresses for corresp ondence

M Thomas Dept of Computing Science B Ormsby Dept

of Aerospace Engineering University of Glasgow Glasgow

G QQ Scotland

k email muffydcsglaacuk bowendcsglaacu

References

L M Barro ca and J A McDermid Formal Meth

o ds Use and Relevance for the Development of Safety

Critical Systems The Computer Journal