Fundamentals of Robotics Verification

Fundamentals of Robotics Veriﬁcation

Dennis Meckel Technische Universitat¨ Kaiserslautern, Embedded Systems Group [email protected]

Abstract

Robots, especially commercial and autonomous ones, are usually complex cyber-physical systems operating in safety-critical environments. Therefore it is important to prove that these systems do not cause harm. Field tests and simulations are not exhaustive and therefore not sufficient. Formal verification is exhaustive and can therefore prove that a given system satisfies its specification. The paper at hand gives a high-level introduction to formal verification and the tools BRAHMS,PROMELA, SPIN, and UPPAAL.

Contents

1 Introduction 1

2 Structure 2

3 Model Checking 2

4 Temporal Logics 3

5 Brahms, Promela and SPIN 5

6 Uppaal 10

7 Conclusions 17

8 Glossary 18

1 Introduction

Cyber-physical systems are ubiquitous. They are not only supporting safety-critical driver and pilot decision with alerting functions or ﬁne-grained adaptions of human control actions but are also striving to conquer the world as fully autonomous mobile robots. [7] Robots are leaving their known environments to explore and work where no robot (or man) has gone before. Highly specialized robots like pavers, road rollers and lorries will team up to extend and maintain the infrastructure used for transportation. [6] Autonomous cars will supersede the need for human drivers. Service robots will assist the elderly and incapacitated in their homes and will also ensure that humans working in hazardous environments remain safe if the robots are not executing the dangerous tasks themselves. [17] 2

The previously mentioned robots are complex systems consisting of tremendous amounts of concur- rently running software modules and hardware components. They have to adapt to unforeseen situations while ensuring that “something bad will never happen”1 For instance, robots should not harm themselves, other robots and most importantly not humans. Field tests and simulations can increase the trust and con- fidence in robots but only verification can proof that robots follow their specifications by systematically and automatically exploring all possible actions.

2 Structure

The adoption of formal verification is motivated by showing the limits of field tests and simulations in section 3. Afterwards, the concept of model checking is introduced as a solution. The later introduced tools and examples (see Table 1) utilize property checks formulated with the temporal logics CTL and LTL. Therefore a high-level introduction to temporal logics is given in section 4. section 5 presents the tools BRAHMS,PROMELA, and SPIN. BRAHMS is used in this section to model a service robot and additional agents including the house in which the robot operates and humans interacting with the robot. As BRAHMS is lacking a verification tool the model gets exported to PROMELA and then verified with SPIN. The tool UPPAAL and the behavior-based robot architecture IB2C are introduced in section 6. Subsequently, the section also shows how to model and verify IB2C networks with UPPAAL. The paper closes with the conclusions given in section 7 and a short glossary in section 8.

Section Tools Veriﬁcation Logics

5 BRAHMS,PROMELA and SPIN LTL

7 KEYMAERA,KEYMAERA X Differential Dynamic Logic (dL)

6 UPPAAL CTL

Table 1: The veriﬁcation tools BRAHMS,PROMELA, SPIN and UPPAAL are introduced in dedicated chapters. KEYMAERA and KEYMAERA X are referenced in the conclusion.

3 Model Checking

Field tests and simulations can be used to reconstruct, analyze and show the presence of undesirable behaviors. Both techniques can also be used for testing speciﬁc use cases. The problem with these measures is that they do not scale well with respect to complexity and that they are too costly (time and budget wise, if possible at all) to explore all possible actions and consequences. Safety-critical systems can’t be trusted and shouldn’t be used without exhaustive checks due to the number of possible unforeseen and undesired consequences. Program testing can be a very effective way to show the presence of bugs, but is hopelessly inadequate for showing their absence. — Edsger W. Dijkstra

1This is a reference to section 4). 3

Exhaustive checks can be realized and automated with the help of formal verification. Model or property checking is such a technique for automatically testing whether a system model meets a given specification. The first step is to create a simplified system model which represents a real-world model, e.g. a microcontroller or a robot. Although the simplification is necessary to keep the model complexity low, the system model has to be as accurate as possible. It is common to reduce data types to binary types. One example is given in section 6 where integer signals representing the degree of behavior activations in [0,1] are reduced to binary signals {0,1}, either representing that a robot behavior is fully inactive or fully active. [3] Such a simplification can drastically reduce the resulting complexity. In the worst case, every possible value has to be checked. The system model is usually realized as an automaton or a collection of interconnected automata (see the IB2C example in section 6). It is also possible to create high-level models which have to be converted into another model for verification (see section 5 which utilizes BRAHMS models, that are automatically converted to SPIN for verification). The specification properties are modeled in the second step. Safety properties state that their property has to hold for all computations and times, e.g. a robot does not collide with a static obstacle. Liveness properties state that some desired state can eventually be reached, e.g. a robot behavior gets activated. Persistence properties are similar to safety properties except that the properties are allowed to not hold up to a point of time. After that point of time, the properties have to hold forever. Fairness (or recurrence) properties hold infinitely often, e.g. a robot behavior gets activated infinitely often. This does not imply that the behavior has to be active all the time. [3] [16] [18] The last step is to execute the verification tools and, depending on the complexity of the system model and the properties, return some hours later to inspect the results. As the number of states becomes easily prohibitively large, verification tools and users have to fight state-explosions by reducing the complexity of system models and by reducing the number of checked states per property. [10] for instance reduces a robot’s safety layer with more than 450 interconnected behaviors to 14 behaviors of interest (with at least 70 automata according to the automatically applied modeling, see section 6) to tackle state explosions and to check one specific property. Despite this huge reduction the verification still takes three hours and 75 GB of memory. [9] Verification tools like SPIN and UPPAAL provide traces for every failed check. This allows to analyze errors in the implementation and specification. It is important to note that the verification of a system does not guarantee that the system is free of errors. A formal verification only shows that the formal system model satisfies the given formal specification. [16, p. 8] Therefore it is crucial to assure that the specification is correct and the system model is representing the real-world system. As a consequence, formal verification should be complemented with simulation-based and real-world tests. [18]

4 Temporal Logics

Temporal logics extend predicate logic with special temporal operators. With these operators they are able to describe the ordering of events in time without introducing time explicitly. [2] Additionally, temporal logics have been developed to support arbitrarily nested formulas. [16] Intuitive formula nesting allows to define specifications in a human readable structured way which improves the readability and understandability compared with mostly flat and unstructured ω-automata transition diagrams. This is especially true when working with bigger automata (ten or more states are already enough to question transition diagrams). [16, Chap. 5] The paper at hand focuses only on the details which are necessary to understand the concepts of the later introduced practical verification techniques, tools and examples. 4

Therefore the following paragraphs contain only a superficial introduction to Linear Temporal Logic (LTL) and Computational Tree Logic (CTL). An in-depth introduction to formal verification and especially temporal logics is given by [16]2. The introduced techniques are based on automata. These automata consist of states (or locations when working with UPPAAL) and connections between the states. Firing an edge (following a connection between two states), which is only possible when the edge is enabled (an edge specific propositional formula evaluates to true), can but does not have to consume time. These automata can be verified with temporal operators and path quantifiers by specifying properties which have to hold, e.g. a robot should never crash into an obstacle. [16, Chap. 5] Properties are checked by inspecting the automata. Path quantifiers quantify over computation paths starting from an initial state s. Given a formula ϕ, the existential path quantifier Eϕ holds when there exists at least one path starting from s which satisfies ϕ. The universal path quantifier Aϕ on the other hand holds only when all possible paths starting from s satisfy ϕ. According to [16, p. 285] LTL does not support the existential operator E. Four (plus one without a path quantifier) graphical examples are depicted in Figure 1. [16, Chap. 5] CTL and LTL support several temporal operators. Some are syntactic sugar and can be represented by a combination of other operators. The work at hand introduces only the unary operators G, F and X used by [18] (see Table 2 and section 5). More operators, especially the expressive binary ones, are described in [16, Chap. 2] and [16, Chap. 5]. The requirements or properties of Table 3 can be formulated with the modal operators of Table 2.

Textual Symbolic Description Gϕ ϕ Globally: ϕ has to hold now and at all points in the future Fϕ ϕ Finally: ϕ has to hold now or at some point in the future Xϕ 3 ϕ NeXt: ϕ has to hold in the next state # Table 2: Basic unary LTL operators introduced and used by [18]. These operators are also available in CTL and of course in CTL* [16, Chap. 5], which has been obtainted by combining CTL and LTL.

Property Example Description Safety ¬ bad Something bad never happens Liveness good Something good eventually happens 3 Fairness send ⇒ receive If one thing occurs inﬁnitely often so does another 3 3 Table 3: Well known properties with examples. [16, Chap. 1] [18]

2[16, p. 280] considers “propositional temporal logics with a discrete model of time”. The later in section 6 introduced UPPAAL uses a continuous time model. [5, p. 206] The differences are not discussed in the work at hand. A Tutorial on Uppaal 5 205

A[]ϕ E<>ϕ ψ� ϕ

ϕ ϕ

A<>ϕ E[]ϕ

Fig. 3. Path formulae supported in Uppaal. Thefilled states are those for which a Figure 1: Path quantifier examples which represent the formulae supported by the CTL based verifica- given state formulaeφ holds. Bold edges are used to show the paths the formulae tion tool UPPAAL. [5] The green colored states satisfy the given formulae. A ϕ (ϕ has to hold now evaluateand at on. all points in the future on all reachable paths) is only satisfied when all states reachable from the initial state satisfy ϕ, therefore all states are green. E ϕ instead needs only one path which satisfies this couldϕ, for instance be a simple the one with expression, the cycle. If likegetsi==7 replaced with, that, then is onlytrue one in state a state somewhere whenever on at i equalsleast one 7. path The (E syntaxϕ) or allpaths of state (A ϕ formulae) has to satisfy isϕ ato superset satisfy3 the formula. of that To satisfy of guards, the formula i.e., a ψ ϕ a state satisfying3 ψ has to be followed3 by a state satisfying ϕ. state formula is a side-effect free expression, but in contrast to guards, the use of disjunctions is not restricted. It is also possible to test whether a particular 5 Brahms, Promela and SPIN process is in a given location using an expression on the form P.l, whereP is a processThe following andl is paragraphs a location. give a short introduction to SPIN and PROMELA before introducing BRAHMS, Inwhich Uppaal utilizes both, deadlock and provides is a high-level expressed abstraction using for a modeling special both state human formulaand robotic activities (although using rational agents. [17] Following this, a real world example is presented based on the work of [18]. this isThe not “SPIN strictly Model a Checker” state formula). can be used toThe verify formula the correctness simply of concurrent consists systems of the by apply-keyword deadlocking logic and model is checking satisfi (ored property for all model deadlock checking) states. to exhaustively A state analyze is a system deadlock models. state The if theresystem are models no outgoing used for analysis action are simplified transitions but essential neither characteristics from the preserving state representations itself or any of of its delayreal-world successors. systems, e.g. Due algorithms to current with complex limitations data types. SPIN in Uppaal exhaustively, the analyses deadlock all possible state executions to verify if the given properties, which can be derived from requirements, hold. If a property formuladoesn’t can hold onlya counter-example be used is with returned reachability by SPIN for manual and invariantlyanalysis and error path correction. formulae [8] [18] (see below).It is not only possible to verify systems as SPIN’s graphical user interface, XSPIN, can visualize simulation runs while developing the system. The authors of [8] are using the term debugging for simulation Reachabilityruns as “no amount Properties. of simulation can Reachability prove the facts we may properties be interested are in; only the a verification simplest run form can of do so”. properties. They ask whether a given state formula,ϕ, possibly can be satisfied by any reachable state. Another way of stating this is: Does there exist a path starting at the initial state, such thatϕ is eventually satisfied along that path. Reachability properties are often used while designing a model to perform sanity checks. For instance, when creating a model of a communication protocol involving a sender and a receiver, it makes sense to ask whether it is possible for the sender to send a message at all or whether a message can possibly be received. These properties do not by themselves guarantee the correctness of the protocol (i.e. that any message is eventually delivered), but they validate the basic behaviour of the model. We express that some state satisfyingϕ should be reachable using the path formulaE✸ϕ. In Uppaal, we write this property using the syntax E<>ϕ. 386 R. Stocker et al. 6 robots, andAs agents. the name Within SPIN was this,originally we chosen use as importantan acronym for “Simple work P onROMELA high-levelInterpreter”, modelling it is of logical to write programs and processes in the language PROMELA to verify if properties hold. It is worth human-agent-robotnoting that according teamwork to [8, Chap. that 2] “SPIN has already has outgrown been at least carried two of these out three using descriptive the Brahms terms, framework [5].but Thus, the name we has assumestuck”. that the key interactions and behaviours of any human- agent-robot scenarioPROMELA is another have abbreviation been captured which stands within for “Process/Protocol a Brahms Meta-Language” model. We and also provides assume that the following control flow constructs: case selection, repetition and unconditional jump. [8] [17] SPIN a set of informalconverts the requirements PROMELA specifications, have which been are constructed. interpreted as LTL The (linear work temporal described logic) properties, in this paper essentiallyto describes Buchi¨ automata. the B uchi¨solid automata arrows are finite within automata Fig. over 1. infinite input sequences. The temporal requirements are then checked by running the automata in parallel and by checking all possible runs.

Human-Agent- Informal Robot Teamwork Match? Requirements Scenario

Automated, Formal Exhaustive, Requirements Formal Veri�cation in Promela in Spin

Brahms Teamwork Model Promela Representation of Teamwork Model Java Representation of Semantic Structures Alternative veri�cation systems [Future Work]

Fig. 1. Overview of the processes described in this paper Figure 2: Process of translating a BRAHMS model to PROMELA for verification with the logic model checker SPIN. Picture and BRAHMSTOPROMELA tool by [17]. Thus, given a Brahms model of a human-agent-robot scenario, we use the formal Because this paper focuses on robot verification it seems natural to utilize tools which describe the semanticsbehaviors described of autonomous in [6] to agents generate with own beliefs a Java and goals. representation BRAHMS is a tool of suited the for semantic this task as structures relevant toit thisis a relational scenario. multi-agent We modeling, then translate simulation and these development structures environment. into EveryPromela agent’s belief process de- depends on detectables which, after detection, get added as facts to the agent’s belief base. This can scriptions,change which how representan agent will respond partial to specific instantiations situations by choosing of the an semantics,action from its executable suitableactivi- for input to the Spin tiesmodelas response. checkerWorkframes [7].are In used parallel, to represent we sequences translate of events the required informal to complete requirements a task while (where thoughtframes are describing reasoning process. As actions in the real-world are not instant BRAHMS possible) toincorporates Promela a time line code of events representing and belief changes. these properties. Finally, we apply the Spin model checker to the Promela descriptions and feed the results back to the high-level scenarios for evaluation. This, then, provides a mechanism for formally verifying properties of human-agent-robot teamwork scenarios. Our paper describes this framework and exhibits its use on a specific domestic scenario, where a helper robot and a house agent work together with a person to to monitor, remind and assist a person with their daily activities in a home environment.

2 Background

2.1 Brahms Brahms is a multi-agent modelling, simulation and development environment devised by Sierhuis [5] and subsequently developed at NASA Ames Research Center. Brahms is designed to model both human and robotic activity using rational agents. Rational agents can be seen as autonomous entities, able to make their own choices and carry 7

As BRAHMS has no built-in formal verification tool [17] [18] BRAHMS models have to be tested with external verifiers. The BRAHMSTOPROMELA tool by [17] converts BRAHMS models to PROMELA for model checking with SPIN. Figure 2 displays the process of translating BRAHMS models to PROMELA. The model is converted first to an intermediary Java representation, which can be further translated into a language of a specific verification tool. The figure shows that PROMELA is supported and that other languages could follow in the future. Following this conversion, it is possible to verify the system by running an automated and exhaustive formal verification with the SPIN tool. [17]

High Level Scheduler Example

The following example introduces the work of [18]. The authors utilize the BRAHMSTOPROMELA tool created by [17] to verify the high-level planner/scheduler of their service robot Care-O-bot, which is an robot assistant deployed at the University of Hertfordshire’s Robot House. The house is equipped with more than 50 sensors which provide real-time information on the state of the house (e.g. drawers and refrigerator closed, toilet flushed, someone is sitting on a chair) and the individuals occupying it. The Care-O-bot is a commercially-available service robot which was specially developed for domestic environments by the Fraunhofer IPA. It features a seven degrees of freedom manipulator arm with a three-fingered gripper as end effector. It is furthermore equipped with stereo sensors, LED lights, a tray, localization sensors and a text-to-speech synthesizing module. The robot utilizes ROS (Robot Operating System) as operating system. High-level commands like “raise tray” or “move to location x” can be sent to the ROS script server which interprets them into low-level commands. The authors abstracted the robot control further by defining the robot’s high-level decision making with a set of rules stored in a relational MySQL database. It is important to note that the database stores not only information about the robot but also sensor data about for all other sensors in the robot house. The MySQL dumps in the accompany repository [1] contain all details about the schema, data and rules. Every rule consists of a guard, the separator == and robot actions. The guards are sequences of propositional statements which are valid SQL queries. If a guard holds then the corresponding robot actions RobotAction* are executed. Listing 1 shows a guard which checks whether the sensor values of sensors 500 and 504 are 1. Sensor 500 has the value 1 when the service robot’s tray is raised while sensor 504 returns 1 when the tray is empty. If the tray is open and empty (the guard holds), then the following five robot actions are executed: Care-O-bot’s light turns to yellow, the bot lowers the tray and waits, the light changes to white, the tray sensor’s value is changed to false and as last step the sensor 501’s value3 is set to true.

SELECT* FROM Sensors WHERE sensorId=500 AND value=1 & SELECT* FROM Sensors WHERE sensorId=504 AND value=1 == light,0,yellow tray,0,down,,wait light,0,white,,wait cond,0,500,0 cond,0,501,1 Listing 1: Guard and robot actions RobotAction* of a decision making rule by [18]

3Sensor 501 is not explained in [18]. The MySQL dump in the accompany repository [1] contains possibly contradicting information, e.g. sensor 501 represents the state of the tray. 8

The database contains 31 rules for the Care-O-bot robot. Every rule has to be converted into the BRAHMS multi-agent workﬂow programming language. The ﬁrst step converts all rules “into a more convenient if-then rule representation”, e.g. the abstract Listing 1 gets rewritten to Listing 2.

IF tray_is_raised AND tray_is_empty THEN set_light(yellow) move_tray_and_wait(lowered_position) set_light(white) wait() set(tray_is_raised,false) set(tray_is_lowered,true) Listing 2: The intermediary format which gets translated to Brahms.

According to [18] the if-then format can be straightforwardly translated into the when-do format of BRAHMS. The rules or more speciﬁcally the intermediary if-then rules are converted to BRAHMS workframes with the service robot’s agent. The guards get translated to workframe conditions which have to hold to execute the robot actions which are translated into action lists. workframe wf_lowerTray { repeat: true; priority: 10;

when(knownval(current.trayIsRaised = true) and knownval(current.trayIsEmpty = true)) do{ conclude((current.lightColour = current.colourYellow)); lowerTrayAndWait(); conclude((current.lightColour = current.colourWhite)); waitForLightColourChange(); conclude((current.trayIsRaised = false)); conclude((current.trayIsLowered = true)); } }

Listing 3: The rule of Listing 1 translated into BRAHMS.

As the robot is allowed to lower the tray more than once the workframe in Listing 3 is set to repeat. The priority deﬁnes the importance of the workframe. This information gets crucial when other workframes are eligible for execution at the same time. The conclude() constructs are updating the agent’s belief base. lowerTrayAndWait() and waitForLightColourChange() are custom constructs. The implementation of lowerTrayAndWait for instance just waits for a short period to simulate a time taking action. 9

The paper of [18] introduces another rule where the user visiting the robot house takes a seat and watches television. The service robot will then ask the user whether it should join watching television. The user can respond with one of three options provided via the GUI of a tablet computer: the robot should join watching television, the robot should return to the charging station or the robot should con- tinue its current task. Depending on the user interaction one of three workframes will trigger and update the robot’s belief base and active task. As a real person can act unpredictablethe person agent is implemented with a set of ﬁve workframes which are modifying the belief base dependent on probabilities.

Robot House Veriﬁcation with LTL

After translating all robot house rules into BRAHMS the next step is to model the scenario which will be veriﬁed with SPIN. The scenario deﬁned by [18] lasts from 12 p.m. to 6 p.m. and includes the robot house, the Care-O-bot, one person and an additional campanile agent measuring the time and keeping other agents updated of the current time. The person may choose to sit down and watch TV, move into the living room area or the kitchen, or send the Care-O-bot into the living room area or the kitchen. The person has to take medication at 5 p.m. Based on the scenario high-level requirements are derived and formulated as LTL formulas (see section 4) for logic model checking with SPIN. All beliefs are represented by the belief operator B. If for instance the Care-O-bot believes that x is true, then BCare-O-bot x is true. The following formulas and descriptions from [18] show how the scenario is modeled with LTL properties that are checked with SPIN. The formulas are ordered by complexity and one example was removed due to space constraints. 1. If the person orders the robot to move to the kitchen, then it has to eventually move into the kitchen.

[BCare-O-bot(BPerson guiGoToKitchen) ⇒ BCare-O-bot(location = Kitchen)] 3 “It is always the case that if the Care-O-bot believes that the person has told it to move into the kitchen, then it will eventually move into the kitchen.” [18] 4. The robot has to remind the person at 5 p.m. to take the medicine. The given LTL formula does not guarantee that the person is taking the medicine. BCampanile Clock(time = 5pm) ⇒ BCare-O-bot(medicineDue) 3 “It is always the case that if the time is 5pm, then the Care- O-bot will believe that the medicine is due.” [18] 3. When the person is watching TV the service robot has to move to the living room and has to ask if it should join.

  BRobotHouse sofaSeatOccupied  ∧ BRobotHouse televisionWattage > 10   B (location = LivingRoomSofa) ⇒ Care-O-bot  ∧ BCare-O-bot askedToWatchTV 3 “It is always the case that if the Robot House believes that the sofa seat is occupied, and if the Robot House believes that the television wattage is higher than 10 watts, then eventually the Care- O-bot will move to the living room sofa and ask the person if they want to watch the television with the Care-O-bot.” [18] 10

Robot House Results Although the presented properties look simple and straightforward the resulting veriﬁcation structures are quite complex with more than 600,000 states and a memory consumption greater than 10 GB. It is important to remember that the robot house features more than 50 sensors and more than 30 high-level decision making rules. Furthermore believes and sensor values can change all the time between 12 p.m. and 6 p.m. As Table 4 shows model checking can be resource-intensive. The authors of [18] assume that adding more rules to the service robot would result in SPIN running out of memory. The authors are optimistic that they can reduce the complexity by improving the BrahmsToPromela translator. Reducing the complexity and memory consumption is necessary to further improve the model by adding other robots, tasks and more realistic workframes (e.g. the person agent consists only of ﬁve basic workframes).

Property States Depth Memory (MB) Time (s) Note 1 652,573 46,617 10,132 20.7 2 652,573 46,617 10,132 20.7 Only discussed in [18] 3 746,497 53,009 11,596 30.7 4 652,573 46,617 10,132 20.3

Table 4: Computational resources of the formal veriﬁcation [18]

6 Uppaal

This section first introduces the UPPAAL toolbox which includes tools for modeling, simulation and verification of real-time systems. Subsequently, an example based on [3], [4] and [10] is presented to give an overview how properties of behavior-based robot systems can be verified with the help of UPPAAL. UPPAAL is a model checker based on the theory of timed automata. Every automaton in UPPAAL is either firing an edge or synchronizing with other automata. A timed automaton (L,l0,C,A,E,I) is a finite- state machine (locations L, initial location l0 ∈ L, set of actions A, edges E between locations, invariants to locations I) extended with a set of clocks C.UPPAAL executes these automata in parallel although all clocks progress synchronously. A system model in UPPAAL consists of multiple timed automata with discrete variables. System model properties are checked by and are specified with a query language based on a subset of CTL (see section 4). [5] The formal language is divided into a state formulae for checking whether an automaton is in a specific location and a path formulae for quantifying over paths or traces. The following list summarizes the basic elements of UPPAAL systems. [3] Please have a look at [5] for more in-depth information. • Templates: Each template contains the definition of one automaton • Locations: Locations can be marked as commited or urgent. Time is not allowed to pass when the system is in an urgent location. “A state is commited if any location in the state is commited. A commited state cannot delay and the next transition must involve an outgoing edge of at least one of the commited locations.” [5] 11

• Edges: Edges are not only connecting locations. An edge can be reﬁned with the following labels altering its impact on the network. – Guard: A boolean guard label determines whether an edge is enabled. The expression has to be side-effect free. – Assignment: The assignment label allows to assign new values by specifying a list of expressions with side-effects. – Synchronization: These labels synchronize between several automata through channels. The label of a sending channel is sufﬁxed with “!” while a receiving channel’s label ends with “?”. The expressions have to be side-effect free and evaluate to a channel. UPPAAL does not allow more than one synchronization per edge. [3] solved this limitation by adding commited intermediate states and edges for every additional synchronization. • Channels: A channel is either a binary synchronization or a broadcast one. A sending binary synchronization channel c! synchronizes with a non-deterministically chosen receiving channel c?. A broadcasting channel c! instead synchronizes with an arbitrary number of receiving channels c?. Any receiver channel c? that can synchronize in the current state must do so. c! is also 4 Christopher Armbrustexecuted et when al. there is no receiving channel c? as broadcasting is never blocking [5].

Location_0 �e s �a i1 > 3 i2 = 3 c1? c2! B=(f a, fr,F) r �ı

�u i1 = 0 Location_1 Location_2

Fig. 2. The generalFigure symbol 3: of aA behaviour simple automatonFig. 3. with A simple initial location automatonLocation with three0, urgent lo- location Location 1 and (s: stimulation,i:commited inhibition location vector,Locationa: cations2. The ( edgeLocation(Location0: initial,0, Location Location1)1:has a guard label i1 > 3 and activity vector,r:a target receiving rating, channele: in- synchronizationurgent, c1?Location. (Location2: committed),1, Location a guard2) contains an assignment label put vector,u: outputi1 vector, = 0. Thef a remaining: function edge(i is1> equipped 3), two with updates a sending (i1 channel = 0,i2 synchronization = 3), label c2! and another calculatinga,f r:assignment function calculating label i2 =r, 3. [3]and two channel synchronisations (c1?: F(e,ι): function calculatingu). receiver, c2!: sender). Figure 3 depicts a simple automaton with three locations and edges with assignment and synchronization labels. The next subsection introduces the behavior-based robotics architecture IB2C. behaviour. OtherBehavior-Based connection types Control include Architecture the combination iB2C of the outputs of a number of competing behaviours using a so-called fusion behaviour (see below) or the sequencingThis of behaviours section introduces using the a special work of coordination [3] and some details behaviour about the (see enhancements [4]). of its successor [10]. The iB2C fusionBoth behaviour papers are using combines UPPAAL theto outputs verify behavior-based of other behaviours systems implemented con- with the behavior-based nected to it accordingcontrol to architecture one of threeIB2C possible (“Integrated fusion Behavior modes (maximum, Based Control”) weighted and the robotics framework MCA2- KL. The following paragraphs introduce the concepts of IB2C. A comprehensive discussion about IB2C average, and weighted sum). LetB be a fusion behaviour to whichp com- is available at [13]. IfFusion you are interested in the robotics framework please have a look at its successor peting behavioursB with activitiesa , target ratingsr , and output vectors FINROCInput[15]c and the successorc of IB2C which is calledc RIB2C [11]. The following introduction to IB2C uc are connected.is In based case on of the a documents maximum [3], fusion, [11] and the [13] outputs ofB Fusion are de- ﬁned as follows:u=u s,a = max (ac), andr=r s, wheres = argmax (ac). Fusion c c behaviours have the same interface as basic behaviours. To facilitate the handling of a large number of interconnected behaviours, behavioural groups encapsulate a number of behaviours or further groups and act as new behaviours in a network.

3.2 Finite-State Automata and Uppaal

The Uppaal toolbox (see [6]) provides an integrated environment for the modelling, simulation, and verification of real-time systems, which are represented as networks of automata (see [1]). Its modelling language extends these automata with structured data types, integer variables, and channel synchronisation. Every automaton mayfire an edge separately or synchronise with another automaton. A Uppaal system consists of instantiations of several templates, each of which contains the definition of an automaton. The basic elements of automata are locations, which are connected via edges. Each location can be marked as committed. A committed state cannot delay and the next transition must involve an outgoing edge of at least one of the committed locations. Edges may have several attributes like guards (side-effect free Boolean expressions to determine whether an edge is enabled), updates (assignments), and synchronisations 12

Every IB2C behavior is based upon a common interface which is depicted in Figure 4. A behavior can be gradually enabled by the incoming stimulation signal s or gradually disabled by the incoming inhibition signal i. Every behavior has furthermore two outgoing control signals. The first one is the activity signal a which represents the degree of influence the behavior intends to have on connected behaviors via their stimulation and inhibition ports. The second signal is the target rating r which describes how content the behavior is with its current state. The inhibition i = max j=0,...,k−1{i j} of a behavior B depends on all k incoming inhibitions from behaviors inhibiting B. Besides activity a a behavior can also pass derived activities ai ≤ a to other behaviors. The activity vector ~a encapsulates the activity signal a and the derived activity signals ai as ~a = (a,a)T . The influence of a behavior is limited by the activation ι = s · (1 − i). From this it follows that a ≤ ι and all derived activities ai ≤ ι. All behavior (or meta) control signals are limited to [0,1]. Data signals are not limited to [0,1] as they are allowed to contain arbitrary data types. The number of data input and output signals depends on the data required or created by the behavior. Incoming data signals are received via the input vector ~e while outgoing data signals are represented by the output vector ~u. The transfer function F calculates the output vector ~u = F(~e,ι) based upon the data input T vector ~e and the calculated activation value ι. fa calculates the activity vector ~a = (a,~a) = fa(~e,ι) and f is responsible for the target rating r = f (~e). The interested reader is referred to [11] which states r 4 Christopherr Armbrust et al. that RIB2C reduces the complexity and implementation overhead by using scalar values for all behavior control signals and automatically calculating the target rating r based upon the activity a. Location_0 �e s �a i1 > 3 i2 = 3 c1? c2! B=(f a, fr,F) r �ı

�u i1 = 0 Location_1 Location_2 ~ Figure 4: Graph representationFig. 2. of The a behavior generalB. symbol The incoming of a behaviour meta signals s andFig.i are 3. A placed simple onthe automaton with three lo- left. Incoming data signals ~e are entering the behavior from above. The activity function f and target (s: stimulation,i: inhibition vector,a: cations (Locationa 0: initial, Location 1: rating function f are calculating the outgoing meta signals ~a and r on the right. The leaving data signals r activity vector,r: target rating,e: in- urgent, Location 2: committed), a guard ~u are calculated by the transfer function F. [3] put vector,u: output vector,f a: function (i1> 3), two updates (i1 = 0,i2 = 3), An IB2C behavior cancalculating stimulate othera,f behaviorsr: function by connecting calculating its activityr, signaland to two their channelstimulation synchronisations (c1?: signals. It can also inhibitF other(e,ι): behaviors function by calculating connecting theu). activity with inhibitionreceiver, signals. c2!: It sender). is not allowed to create circles by connecting activity signals to stimulation or inhibition signals. Data signals can be connected as long as the signals and their directions are compatible. [13]. Behaviors can be encapsulated inside behavioral (or behavior) groups. As all behaviors are sharing the same interface groupsbehaviour. are treated like Other any other connection behavior. The types same applies include for I theB2C combination fusions which of the outputs of a are used for arbitration ofnumber competing of behaviors. competing A fusion behaviours is connected using to the activities a so-calledac, target fusion ratings behaviour (see below) rc and output vectors uc ofor the thec sequencingto be fusioned of behaviors. behaviours The result using of a the special fusion depends coordination on the behaviour (see [4]). chosen fusion method (maximum, weighted average or weighted sum). A maximum fusion calculates The iB2C fusion behaviour combines the outputs of other behaviours con- s = argmaxc(ac) with the purpose to let the behavior with the highest activity win the competition. The fusion will then publish thenected values tou = itus according, a = as and tor = oners. [3]of three The notations possible between fusion [3] modes and [10] (maximum, weighted differ in details (compare Figureaverage, 4) and and Figure weighted 5, e.g. F sum).in [3] is LetequalB toFusion~F in [10].be a fusion behaviour to whichp com-

peting behavioursB Inputc with activitiesa c, target ratingsr c, and output vectors uc are connected. In case of a maximum fusion, the outputs ofB Fusion are de- ﬁned as follows:u=u s,a = max (ac), andr=r s, wheres = argmax (ac). Fusion c c behaviours have the same interface as basic behaviours. To facilitate the handling of a large number of interconnected behaviours, behavioural groups encapsulate a number of behaviours or further groups and act as new behaviours in a network.

3.2 Finite-State Automata and Uppaal

(a) The general symbol of a behaviour (s: stimulation, i: (b) The symbol of a fusion behaviour forn 2 ⃗ c = inhibition vector, a: activity vector,r: target rating, e: competing behaviours. ⃗Figure 5: The depicted⃗ fusion behavior BFusion fusions two competing behaviors. This implies that input vector, u: output vector, f : function calculating a, ⃗ ⃗a ⃗ f : function calculatingther, inputF e,ι vector: function~e calculatingof BFusionu). is connected to two output vectors ~ui, two target ratings ri and two activity r ⃗ ⃗ ⃗6 Christopher Armbrust et al. signals ai. [10] Fig. 1. The symbols of two basic iB2C behaviours.

i0_changed? Modeling iB2C behaviors with Uppaal i_changed_internal! i_value=0, processing = false processing = true [3] encapsulates every IB2C behavior with five templates. Each template is focused on oneUninhibited behavior Inhibited_by_First signal or calculation: StimulationInterface, InhibitionInterface, ActivationCalculation, ActivityCalculation, and TargetRatingCalculation. The encapsulation and specializationi0_changed? on i_changed_internal! Fig. 3. Representation of the different layers of sensor data processing. i_value=1, processing = false one behavior signal allows to reuse templates for other behaviors and reduces the number of statesprocessing per = true automaton which simultaneously(ActivityCalculation increases the readability.) indicates Synchronization whether a behaviour channels isinterconnect cur- the Fig. 2. A simple automaton with three locations (Location_0: initial, five instantiated templates per behaviorrently active and or also inactive. behaviors For state with spaceeach other. reduction, To further all behaviour reduce the com-i_changed_internal! Location_1: normal, Location_2: committed), a guard (i1 > 3), two updates i1_changed? (i1 0, i2 3), and two channel synchronisationsplexity (c1?: the receiver, possiblec2!: sender). meta signalsignals values have[0,1] beenare reduced reducedto from{0, the1} intervalwhere 0[0 is, 1 “inactive”] to thei1_changed? set and0, 1 is. “active”.processing = false = = The interaction between the automata of one behaviouri_value=1, as well{ as} processing = true The symbol of a maximum fusion behaviour is depicted in Fig. 1(b). of different behavioursUnstimulated is realised using synchronisation channels. For the definition of specific sequences inside the rcs, the condi- As Uppaal allows only one synchronisation per edge, committed tional behaviour stimulator(cbs) is used. It gets active when the statess_changed? are added to receive and send multiple signals together.i_changed_internal! The connected signals from other behaviours fulfil the assigned con- models_value=1, of a complete behaviours_changed_internal! network is built by anprocessing automated = false processing = true processing = false ditions. For more information about cbs see [29]. process that creates the automata with the correct connections. i1_changed? The iB2C allows for encapsulating several behaviours in so- Here, the channel names used in the templates are automatically i_value=0, called behaviour groups, which act as new behaviours in the renamed to ensure that every set of automata reacts only on its processing = true i1_changed? network and provide the same interface as that of a standard appropriate signals. Detailed information about the modelling of i0_changed? behaviour. Hence, groups can be combined to larger groups. This s_changed_internal!iB2C networks as networkss_changed? of synchronised automata can be found allows for building hierarchical behaviour networks. processingin [5]. = false s_value=0, processing = true Uppaal’s verifier is used to check certain properties of the gen- Inhibited_by_Second Inhibited_by_Both 3.2. Formal modelling and verification erated models. It can also provide traces to witnesses and coun- terexamples. TheStimulated Uppaal toolbox is used because of its powerful i0_changed? The Uppaal5 toolbox (see [30]) is used for modelling and and easy to use gui as well as because of its xml-based file for- verifying the behaviour network. A UppaalFigure system 6: StimulationInterface is built up of a set mat, whichof a single facilitates behavior the automaticB. [3] The generation receiving of channel Uppaals mod-changed? can of automata, each of which is a parametrisedbe influenced instantiation by another of a so- behavior.Fig.els. 4. However, If sStimulationInterfacechanged? the manualgets inputtriggered, of complex the s value queriesof value isFig. tedious toggles 5. and InhibitionInterface between 0 for a behaviour called template. Automata consist of locationsand 1. Theand edges sending connecting channelas singlechangederror-prone, behaviour.internal! therefore,is [9] used describes for calculating a concept theof a new graphic-basedthat activation is inhibitedι of B by by two others. them. Locations can be marked as committed or urgent. When user interface that allows the user to easily define queries, which the system is in an urgent locationa it receiving cannot delay. channel Committed in the ActivationCalculationare then automatically transformedtemplate. to so-called observer automata locations are even more restrictive. As long as at least one and integrated into the model of the network. Current and future automaton of a system is in a committedThe location,StimulationInterface time does worktemplate deals and is will depicted deal with inextending Figure 6. the It featuresgraphical two interface regular in locations not pass and the next transition mustUnstimulated involve an outgoingand Stimulated edge a waywhich that are the showing techniques whether presented a behavior here areB supported.is stimulated One or ad- is not stimu- of at least one of the committed locations. An edge can have a dition in the context of this work is the possibility for the user to guard (side-effect free Boolean expressionlated. to The determine stimulation whether state is alsographicallyThe represented automaton design by queries the s thatvalue modelling involvevariable. sensor If themodels. the automaton inhibitory is in location inputs of a behaviour is built up an edge is enabled), updates (assignments),Unstimulated and a synchronisationand receivesin. an as similarchanged signal, fashion. then s value Theis mostset to 1 and notableprocessing diffiserence set to is that there can possibly Guards are side-effect free Booleantrue expressions. The automaton to determine will then3.3. move Integration to the committed of hardware and information unnamed into intermediary the model state on the left. This whether an edge is enabled. Updatesstate assign will values be left to immediately variables.be by more taking the than edge leading one inhibitoryto the Stimulated input.location Hence, which sends there the is actually not one single Synchronisations of automata are realised using channels, where The paper at hand extends the existing modelling approach ‘‘!’’ and ‘‘?’’ are used to mark them as sending or receiving,InhibitionInterfaceby integrating the possibility to, model but the for occurrence each number of sensor of inhibition inputs, a special tem- respectively. Fig. 2 depicts an example automaton. platefailures. is For created this, it is required automatically to adapt the formal during modelling of the the modelling process. Figure 5 exem- Each behaviour within an iB2C network is represented by behaviours that react on sensor inputs. These special behaviours a set of five finite-state automata. These automata deal withplarilywill be shows called safety a behaviours template in thefor following. two inhibiting behaviours. The main locations are the input signals (stimulation and inhibition), the output sig- According to a concept for sensor processing developed at the nals (activity and target rating), and the calculation of the acti- UninhibitedRobotics Research, Inhibited Lab, the connectionby ofFirst real sensors, Inhibited to their by Second, and Inhibited by- vation of the behaviour. For example, one of the five automata Bothcorresponding. Similar safety to behaviours the StimulationInterface is done using different layers , the automaton updates its vari- for processing the sensor data (see Fig. 3). The first layer is ablesresponsible and noti for thefies sensor other data preprocessing automata using ofB grid about maps changes. 5 http://www.uppaal.org/. forIn data an storage. iB2C One behaviour, grid map is produced stimulation for every hardwares and inhibitioni are combined to the activationι. This is modelled by ActivationCalculation (see Fig. 6). The modelling of activity and target rating is done in an extremely simple form, as this is sufficient when focussing on the interaction of behaviours. Figure 7 shows the automaton of ActivityCalculation. It switches between Inactive and Inactive but Activated depending on the signal iota changed. In the latter of the two states, it can switch to Active (and back) arbitrarily. The calculation of the target rating is modelled in a similar way by TargetRating- Calculation (see Fig. 8). Of course, it is possible to model the calculation of these two behaviour signals in a more realistic fashion if this is necessary for the verification process and enough information about the calculation functions is available. How this is done for a fusion behaviour will be briefly described below. As mentioned above, the modelling approach currently limits behaviour signals to 0,1 . Hence, instead of the real formulae for activity and target rating of a{ fusion} behaviour (see Section 3.1), a simpler calculation is used: The activity of a fusion behaviourB Fusion can only be above 0 ifB Fusion is ac-

tivated and at least one of its input behavioursB Inputc is active. Similarly, 14

s changed internal signal to inform other automata of B that the stimulation s changed. It also re- sets the processing value by setting its value to false. If the automaton is in location Stimulated and receives s changed, s value is set to 0 and processing is set to true. Immediately leaving the committed intermediary state on the right leads to sending the s changed internal signal and setting s changed internal to false. The implementation of the InhibitionInterface is similar to the implementation of the Stim- ulationInterface. As IB2C supports multiple incoming inhibition signals per behavior it is necessary to create specialized implementations depending on the number of incoming signals. The implementation for one inhibition signal is analog to the implementation of the previously introduced Stimulation- Interface. Implementations for multiple inhibition signals are more complex due to the number of inhibitions and states. The different inhibition templates are generated automatically by [3]. Figure 7 shows an implementation with two inhibition signals and four non-commited locations Uninhibited, Inhibited by First, Inhibited by Second and Inhibited by Both. Although the model is restricted to {0,1} signal values the calculation of the inhibition value i = max j i j remains unchanged. If at least one inhibition signal is 1 then the behavior is fully inhibited (i value = 1, see incoming edges of Inhibited by First and Inhibited by Second). If both inhibition signals are active the automaton will be in the Inhibited by Both location. Leaving this location, which happens only when one inhi- 6 Christopher Armbrustbition signal et gets al. inactive, will not change the inhibition status i value. If the second inhibition signals turns inactive, then i value gets set to 0.

i0_changed? i_changed_internal! i_value=0, processing = false processing = true Uninhibited Inhibited_by_First

i0_changed? i_changed_internal! i_value=1, processing = false processing = true

i_changed_internal! i1_changed? i1_changed? processing = false i_value=1, processing = true Unstimulated s_changed? i_changed_internal! s_value=1, s_changed_internal! processing = false processing = true processing = false i1_changed? i_value=0, processing = true i1_changed? i0_changed? s_changed_internal! s_changed? processing = false s_value=0, processing = true Inhibited_by_Second Inhibited_by_Both Stimulated i0_changed?

Figure 7: Implementation of an with two incoming inhibition signals. [3] Fig. 4. StimulationInterface of Fig. 5. InhibitionInterfaceInhibitionInterface for a behaviour As IB2C supports multiple incoming inhibition signals per behavior it is necessary to create for every a single behaviour. number of incomingthat signals is an inhibited own template. by Futuretwo others. work based on RIB2C will only need one template as RIB2C allows at max one inhibition signal per behavior. [11]

The automaton modelling the inhibitory inputs of a behaviour is built up in a similar fashion. The most notable difference is that there can possibly be more than one inhibitory input. Hence, there is actually not one single InhibitionInterface, but for each number of inhibition inputs, a special template is created automatically during the modelling process. Figure 5 exem- plarily shows a template for two inhibiting behaviours. The main locations are Uninhibited, Inhibited by First, Inhibited by Second, and Inhibited by- Both. Similar to the StimulationInterface, the automaton updates its variables and notifies other automata ofB about changes. In an iB2C behaviour, stimulations and inhibitioni are combined to the activationι. This is modelled by ActivationCalculation (see Fig. 6). The modelling of activity and target rating is done in an extremely simple form, as this is sufficient when focussing on the interaction of behaviours. Figure 7 shows the automaton of ActivityCalculation. It switches between Inactive and Inactive but Activated depending on the signal iota changed. In the latter of the two states, it can switch to Active (and back) arbitrarily. The calculation of the target rating is modelled in a similar way by TargetRating- Calculation (see Fig. 8). Of course, it is possible to model the calculation of these two behaviour signals in a more realistic fashion if this is necessary for the verification process and enough information about the calculation functions is available. How this is done for a fusion behaviour will be briefly described below. As mentioned above, the modelling approach currently limits behaviour signals to 0,1 . Hence, instead of the real formulae for activity and target rating of a{ fusion} behaviour (see Section 3.1), a simpler calculation is used: The activity of a fusion behaviourB Fusion can only be above 0 ifB Fusion is activated and at least one of its input behavioursB Inputc is active. Similarly, 15

Template ActivationCalculation models the calculation of the activation ι = s · (1 − i) (see Fig- ure 8). Therefore channels and values from StimulationInterface and InhibitionInterface are used. If the behavior is inactive (ι = 0), then the automaton is located in the Inactivated state. The behavior is only active in this model when s value == 1 && i value == 0 as stimulation s > 0 (especially s = 1 in the simpliﬁed model) is required to have the chance to get active and inhibition i = 1 is fully inhibiting the behavior even if s Veri= 1. ﬁcation of Behaviour Networks Using Finite-State Automata 7

Inactive Inactive_but_Activated iota_changed?

iota_changed? Inactivated iota_changed! processing = false a_changed! a_value = 0, a_changed! processing = false a_value = 1

(s_value==0) || (i_value==1) a_changed! processing = false a_value = 0 s_changed_internal? iota_changed? processing = true i_changed_internal? processing = true processing = true

i_changed_internal? Active processing = true s_changed_internal? processing = true Fig. 7. ActivityCalculation (s_value==1) && (i_value==0) (calculating the activity). iota_changed! processing = false r_changed! Activated r_value = 0 Satisfied Unsatisfied FigureFig. 8: 6. The ActivationCalculation modeled activation ι valueis (calculating limited to {0,1 the}. [3] r_changed! activation from the stimulation and the inhibi- r_value = 1 tion). Fig. 8. TargetRatingCalculation (calculating the target rating). The ActivityCalculation and TargetRatingCalculation templates are “extremely simple” [3] but still sufficient for verifying behavior interactions. Instead of modeling behavior specific transfer functions rough and generic approximations are modeled. It is possible to model these calculations in a more realistic fashion if required. it can only be unsatisfied with the situation if at least one of its input be- Both templates arehaviours lacking guard also labels. is. These Hence, reachable considerations edges without lead a synchronization to FBActivityCalculation label can with 18 be fired arbitrarily. Ifand the activation FBTargetRatingCalculationι changes to 0 the ActivityCalculation with 8 locationsautomaton (in depicted the case in of two connected Figure 8 moves to the initial location Inactive. If the iota changed signal is received the automaton input behaviours). The most relevant locations are Active 01 (onlyB Input0 ac- moves to the Inactive but Activated location. Now it is possible to arbitrarily fire the edge leading tive), Active 10 (onlyB Input1 active), Active 11 (both active), and Inactive to the Active location(none and vice active). versa. If Dueiota tochanged the lackfires againof space, the automaton these automata will move back cannot to the be depicted here. Inactive location as the value of the activation ι limits the effective activity a ≤ ι. The creation of the Uppaal models is done automatically from a bbs within The TargetRatingCalculationa running mca2-kltemplate isprogramme. even simpler. During It contains the only model the two creation, locations the synchronisa- Satisfied and Unsatisfiedtion channels. It arbitrarily between fires the the currently automata available are edge established: which also updates When the a template is in- r value variable andstantiated sends the r changed for behavioursignal. B, it is provided with the correct channels depending on the connections ofB with other behaviours. For example, if a behaviour B0 stimulates a behaviourB 1 (see Fig. 9), thens changed of the instance of StimulationInterface belonging toB 1 is set toa changed of the instance of ActivityCalculation ofB 0. Hence, when ActivityCalculation ofB 0 sends a changed, StimulationInterface ofB 1 synchronises with the former.

3.4 Veriﬁcation of Behaviour Network Models

As soon as a bbs has been modelled as a network offinite-state automata, Uppaal’s verifier can be used to check certain properties of the model. By this means, it is possible to gain information about when a behaviour can get active, whether a set of behaviours can be active at the same time etc. How this is done Verification of Behaviour16 Networks Using Finite-State Automata 7

Veriﬁcation of Behaviour Networks Using Finite-State Automata 7 Inactive Inactive_but_Activated iota_changed?

Inactive iota_changed?Inactive_but_Activated Inactivated iota_changed! iota_changed? processing = false a_changed! a_valueiota_changed? = 0, a_changed! Inactivated iota_changed! processing = false a_value = 1 processing = false a_changed! (s_value==0) || (i_value==1) a_value = 0, a_changed! processing = false a_valuea_changed! = 1 processing = false a_value = 0 s_changed_internal? (s_value==0) || (i_value==1) iota_changed? a_changed! processing = true processingi_changed_internal? = false processing = truea_value = 0 s_changed_internal? processing = true iota_changed? processing = true i_changed_internal? processing = true processing = true i_changed_internal? Active processing = true i_changed_internal? s_changed_internal? Active processing = true Figureprocessing 9: Rough = true activity a calculation approximation ActivityCalculation which can arbitrarily s_changed_internal? Fig. 7. ActivityCalculation switch between the Inactive but Activated and Activated locations as long as no iota changed (s_value==1) && (i_value==0)processing = true (calculating the activity). iota_changed! signal is received. [3] Fig. 7. ActivityCalculation processing(s_value==1) = false && (i_value==0) (calculating the activity). iota_changed! r_changed! processing = false Activated r_value = 0 Satisfied r_changed!Unsatisfied Activated r_value = 0 Fig. 6. ActivationCalculation (calculating the Satisfied Unsatisfied r_changed! activationFig. 6. ActivationCalculation from the stimulation and (calculating the inhibi- the r_value = 1 r_changed! tion).activation from the stimulation and the inhibi- r_value = 1 tion). Fig. 8. TargetRatingCalculation Figure 10: The target rating functionFig. 8.f rTargetRatingCalculationis usually behavior speciﬁc and can therefore be highly specialized. This TargetRatingCalculation(calculatingtemplate the target by [3] does rating). not represent a specialized target rating (calculating the target rating). function. Instead it arbitrarily changes its contentment between Satisfied and Unsatisfied. [3]

The activity aF of a modeled fusion behavior F can only be greater than 0 (or be equal to 1) when F B a > it can only be unsatisfied withis stimulated, the situation not inhibited if and at at least one one of the of fusioned its input behaviors be-i provides an activity Bi 0. havioursit canalso only is. be These unsatis considerationsfiedItswith target rating the leadr situationF can to only FBActivityCalculation be if unsatisfied at least if atone least of one its of theinput with competing be- 18 behaviors is not content. [3] haviours also is. These considerationsmodeled the templates lead forto aFBActivityCalculation fusion with two competing behaviors. withThe 18 activity calculation template and FBTargetRatingCalculation with 8 locations (in the case of two connected and FBTargetRatingCalculationconsists of 18 with locations. 8 locations The target (in rating the calculation case of two contains connected 8 locations. The resulting automaton is neither depicted in [3] nor here due to the lack of space. input behaviours). The most relevant locations are Active 01 (onlyB Input0 ac- input behaviours). The most relevant locations are Active 01 (onlyB Input0 active), Active 10 (onlyB Input1 active), Active 11 (both active), and Inactive tive), Active 10 (onlyB Input1 active), Active 11 (both active), and Inactive (none(none active). active). Due Due to to the the lack lackVerification of of space, space, of iB2Cthese these networks automata automata cannot cannot be depicted here. here. The creation of the Uppaal models is done automatically from a bbs within The creation of the Uppaal[3] and consequently models is [10] done are automatically automatically exporting from UPPAAL a bbsmodels within from instantiated IB2C networks. a runninga running mca2-kl mca2-kl programme. programme.The previously During During introduced the the modelchannels model and creation, creation, variables the theare prefixed synchronisa- synchronisa- with the name of the corresponding be- tiontion channels channels between between the thehavior automata automata to generate are are unique established: established: identifiers. If When Whenthe activity a oftemplate a behavior is isB0 in-is in- connected to the stimulation or inhibition of another behavior B , then the connected stimulation or inhibition signal of B gets replaced stantiatedstantiated for for behaviour behaviourBB,, it it is is provided provided with with the the1 correct correct channels channels depending 1 on the connections ofB withby the other activity behaviours. signal of B0. In short For outgoing example, behavior if a meta behaviour signals of a behavior B0 are replacing the on the connections ofB withreceiving other behavior behaviours. meta signals of For another example, behavior ifB1 ato interconnect behaviour the automata. B0Bstimulates0 stimulates a a behaviour behaviourBB1The1(see(see minimal Fig. Fig. verification 9), 9), then then exampless changedchanged Figure 11 providedofof the the by instance instance [3] is explained of of in the following paragraphs. StimulationInterfaceStimulationInterface belonging belongingThe example to to containsBB11isis the set set standard to toaa behaviorschangedchangedB0 andofof the theB1. The instance instance activity of of ofB0 is connected with the stimulation of B1. It follows that B1 can only be active when B0 is active. This property is checked with the ActivityCalculationActivityCalculation of ofBB0.0. Hence, Hence, when when ActivityCalculation ActivityCalculation ofofB 00 sendssends a changeda changed, StimulationInterface, StimulationInterface of ofBB11synchronisessynchronises with with the the former.

3.43.4 Veri Veriﬁcationﬁcation of of Behaviour Behaviour Network Network Models Models

AsAs soon soon as as a abbs bbs has has been been modelled modelled as as a a network network of offifinite-statenite-state automata, UppaalUppaal’s’s veri verifierfier can can be be used used to to check check certain certain properties properties of of the the model. By By this this means,means, it itis is possible possible to to gain gain information information about about when when a a behaviour behaviour can get active, active, whetherwhether a set a set of of behaviours behaviours can can be be active active at at the the same same time time etc. etc. How this is is done done 17

query in Listing 4. The query evaluates to false as there is no path where B1 is active while B0 is not active.

E<> (B1_activity_calculation.Active && !B0_activity_calculation.Active) Listing 4: Simpliﬁed stimulation check which ignores the processing ﬂag. [3]

B1 is only able to get active when B0 is active. This follows from the activity limitation a ≤ ι and the complexity reduction a,ι ∈ {0,1}. B1 does not have to be active as it is allowed to be in the Inactive but Activiated state. This trait is checked with the query in Listing 5. UPPAAL ﬁnds a path on which B0 is active and B1 is not active, which evaluates the referenced query to false.

A[] (B1_activity_calculation.Active || !B0_activity_calculation.Active) Listing 5: Simpliﬁed activity check which ignores the processing ﬂag. [3] 8 Christopher Armbrust et al.

B 0

d! d! B1_s_changed?

B0_a_changed! ge ge B0_a_changed?

an an

ch ch

a_ a_ B1_s_changed?

0_ 0_

B B B B0_a_changed? 1

B0_activity_calculation B1_stimulation_interface

Figure 11:Fig. The 9.B activity0 stimulating signalB of1 behaviorwith itsB activity.0 is interconnected Thefilled stimulation with the stimulation input ofB signal0 indi- of behavior B1, thereforecatesB that0 is stimulating it is permanentlyB1. The stimulated. automaton The on the automata right shows show that how the such activity a connection of B0 replaces the between two behaviours is modelled. In StimulationInterface ofB 1,s changed has stimulation signal of B1 in the model. [3] been replaced witha changed ofB 0, which is sent out by ActivityCalculation ofB 0. The attentive reader might wonder where the previously introduced Processing flags are hidden in Listing 4 and Listing 5. [3] left them out due to limited space. These flags have to be considered when checkingshall properties be demonstrated as otherwise usingintermediate the network locations depicted could influence in Fig. 9. the With results. a very simple [3] appliedquery it the can introduced be checked techniques whether toB model1 can be and active check without parts ofB the0 being behavior-based active, too: navigation system ofE<> the (B1 off-roadactivity robotcalculation.ActiveRAVON. The follow-up && paper !B0 activity [10] verifiedcalculation.Active) the safety layers of5 RAVON and the indoor robotAsBMARVINcan only. The be safety stimulated layer of byRAVONB andconsistsa ι (see of more Section than 3.1) 450 behaviors. this query [10] tackles 1 0 ≤ the stateevaluates explosion problemto false. by The slicing assertion and reducing thatB 1 theis behavior always active network ifB to0 a sub-networkis active can of be only those behaviorsfalsi thatfied influence with the the following behavior query: of interest. This leads to a network with only 14 behaviors for their “RR1”A[] safety (B1 activity requirement.calculation.Active The verification of || this !B0 apparentlyactivity smallcalculation.Active) network needs around 3 h and 76 GB. [10]This concludes assertion that “the is incorrect already implemented asB 1 can be techniques stimulated are by notB 0 sufficient, but still to be enable inactive. the complete verificationWhile of large answering behavior such networks”. queries for small networks is trivial, it gets much more complicated in sophisticated systems consisting of more behaviours. 7 Conclusions 4 Example Model checking is a formal verification technique suited for real-time systems [14] [16] although it is possibleIn that this a system section, model a part consists of the of behaviour-based too many states and navigation can therefore system not be used verified to control due to time and 6 memorythe constraints off-road [10]. robot The ravon paper summarizes(see [3]) is two veri techniquesfied against which certain should requirements be combined tous- tackle the ing Uppaal models of behaviour networks. Its purpose is to coordinate several (groups of) behaviours that realise different approaches for calculating the next target coordinates. There are two ways for driving the robot towards a goal: The first implements a direct point access, where the vehicle is guided directly to a target location. The second extends this by additionally specifying the vehicle’s orientation at the target (point access with orientation). The behaviour network called (G) Drive Control (see Fig. 10a) shall coordinate the following three navigation approaches:

A1) A high-level navigation component is able to provide target coordinates (without or including a target orientation) via behaviours (F) Nav. Direct Point Access (DPA) Interface and (F) Nav. Point Access with Orientation (PAO) Interface.

5 Due to the limited space, the check of the Processingflag is left out in the queries. 6 ravon:Robust Autonomous Vehicle for Off-road Navigation 18 problem. First, the system model should be a simplified, but also as accurate as possible, representation of the real world system. It is common to reduce data types drastically, e.g. in section 6 integer typed variables are replaced with booleans ones without drastically changing the semantics. The second remedy is to reduce the system model to states of interest. If it is known that not all states have to be considered for a given property, then the superfluous states should be removed before model checking. This task can be, depending on the model, automatized, e.g. in the case of IB2C networks a simple network traversal can identify predecessor and successor sets [11, Chap. 4], which means that unreachable behaviors (and their modeled states) can be identified and removed or be not modeled at all. The BRAHMS example discussed in section 5 shows that there is not always one tool which solves all verification needs. The authors of [18] store their high-level decision making logic inside a MySQL database. The rules are converted into the BRAHMS language and are afterwards translated into PROMELA for verification with SPIN. It could be advisable to keep such translation chains as short as possible to not run into incompatibilities or unexpected side-effects. On the other hand, verification tools should be verified and documented in formal languages. Robots are not only cyber-physical systems, they are also hybrid systems as they feature discrete and continuous dynamics. While reachability is decidable for real-time systems, it is not decidable for hybrid systems. As consequence, continuous variables have to be represented by clocks instead of physical motion in space, e.g. when working with UPPAAL. [14] This is of course not a problem when the system model is representing aspects not depending on continuous dynamics. The authors of [14] propose the tool KEYMAERA and its successor KEYMAERA X to verify hybrid systems with dL (Differential Dynamic Logic).

8 Glossary

Cyber-Physical System Cyber-physical systems (CPS) are systems which are combining cyber capabilities (e.g. computation, communication) with physical capabilities (physical processes, e.g. motion) to solve problems that neither part could solve alone. [12, L0.1] Logic Model Checking Model Checking (or property checking) is used to automatically check the correctness of a given system model. The system model is a simplified model of a real system, which preserves the essential characteristics but is also significantly reduced in complexity to make exhaustive verification feasible. [8] Hybrid System Hybrid systems feature both discrete and continuous dynamics. The movement of a car represents the continuous dynamics which can be described by differential equations. A discrete controller which for instance influences the speed completes the hybrid system with discrete dynamics. [14] State Explosion According to [16] the “number of states that must be considered for verification grows exponentially with the size of the system”. This leads to “exploding” automata (the number of states increases exponentially) and rapidly rising time and memory requirements. This can lead to the problem that it is no longer feasible to verify a given system model. The problem can be tackled by slicing the automata or simplifying the system model. [10] 19

References

[1] Acceptable robotiCs COMPanions for AgeiNg Years: EU ACCOMPANY Project Packages Repository Avail- able at https://github.com/uh-adapsys/accompany. [2] Christopher Armbrust (2015): Design and verification of behaviour-based systems realising task sequences . [3] Christopher Armbrust, Lisa Kiekbusch, Thorsten Ropertz & Karsten Berns (2012): Verification of behaviour networks using finite-state automata. In: Annual Conference on Artificial Intelligence, Springer, pp. 1–12. [4] Christopher Armbrust, Lisa Kiekbusch, Thorsten Ropertz & Karsten Berns (2013): Tool-assisted verification of behaviour networks. In: Robotics and Automation (ICRA), 2013 IEEE International Conference on, IEEE, pp. 1813–1820. [5] Gerd Behrmann, Alexandre David & Kim G. Larsen (2004): A Tutorial on Uppaal, pp. 200–236. Springer Berlin Heidelberg, Berlin, Heidelberg, doi:10.1007/978-3-540-30080-9 7. Available at https://doi.org/ 10.1007/978-3-540-30080-9_7. [6] K. Berns, B. Jorg,¨ R. Kalmar, P. Liggesmeyer, S. Liu, M. Traub & G. Uhl (2014): Intelligente Nutzfahrzeuge und verlassliche¨ Nutzfahrzeugverbunde¨ . Commercial Vehicle Cluster Sudwest¨ GmbH, Kaiserslautern, Ger- many. [7] Edmund M Clarke, Bruce Krogh, Andre Platzer & Raj Rajkumar (2008): Analysis and verification challenges for cyber-physical transportation systems . [8] Gerard J Holzmann (2004): The SPIN model checker: Primer and reference manual. 1003, Addison-Wesley Reading. [9] Lisa Kiekbusch (2017): Analysis and Verification of Complex Robot Systems using Behaviour-Based Control. doctoralthesis, Technische Universitat¨ Kaiserslautern. Available at http://nbn-resolving.de/urn:nbn: de:hbz:386-kluedo-46162. [10] Lisa Kiekbusch, Christopher Armbrust & Karsten Berns (2015): Formal verification of behaviour networks including sensor failures. Robotics and Autonomous Systems 74, pp. 331–339. [11] Dennis Meckel (2017): Strukturanalyse von Verhaltensnetzen auf Meta-Signal-Ebene. [12] Andre´ Platzer (2014): Lecture Notes on Foundations of Cyber-Physical Systems Available at http://www. cs.cmu.edu/~aplatzer/course/fcps14/fcps14.pdf. [13] Martin Proetzsch (2010): Development Process for Complex Behavior-Based Robot Control Systems. RRLab Dissertations, Verlag Dr. Hut. http://www.dr.hut-verlag.de/978-3-86853-626-3.html. [14] Jan-David Quesel, Stefan Mitsch, Sarah Loos, Nikos Arechiga´ & Andre´ Platzer (2016): How to model and prove hybrid systems with KeYmaera: a tutorial on safety. International Journal on Software Tools for Technology Transfer 18(1), pp. 67–91. [15] Max Reichardt, Tobias Fohst¨ & Karsten Berns (2012): Introducing finroc: A convenient real-time framework for robotics based on a systematic design approach. Robotics Research Lab, Department of Computer Science, University of Kaiserslautern, Kaiserslautern, Germany, Technical Report . [16] K. Schneider (2003): Verification of Reactive Systems – Formal Methods and Algorithms. Texts in Theoretical Computer Science (EATCS Series), Springer. [17] Richard Stocker, Louise Dennis, Clare Dixon & Michael Fisher (2012): Verifying Brahms Human-Robot Teamwork Models. In Luis Farinas˜ del Cerro, Andreas Herzig & Jer´ omeˆ Mengin, editors: Logics in Artificial Intelligence, Springer Berlin Heidelberg, Berlin, Heidelberg, pp. 385–397. [18] Matt Webster, Clare Dixon, Michael Fisher, Maha Salem, Joe Saunders, Kheng Lee Koay & Kerstin Daut- enhahn (2014): Formal verification of an autonomous personal robotic assistant. Proc. AAAI FVHMS , pp. 74–79.