A Conceptual Model of of Werner Damm Alberto Sangiovanni Vincentelli OFFIS University of California at Berkeley Escherweg 2 Department of EECS, 515 Cory Oldenburg, Germany Berkeley, CA 94720, USA +49-441-9722-500 +1 510 642 4882 [email protected] [email protected]

ABSTRACT Finally, the of Swarm has been recently introduced where In this paper, we present the essential features of CPS Systems of many of the aspects quoted above come together. As pointed out Systems (SoS) and we develop a conceptual, rigorous model for in [38], industry observers predict that by 2020 there will be such systems that can support the development of analysis and thousands of smart sensing devices per person on the planet; if so, synthesis tools. We also address issues related to safety critical we will be immersed in a sea of input and output devices that are and secure applications and we outline how to cope with failures embedded in the environment around us and on or in our bodies. of SoS. The concept of wireless sensor networks is not new. Sensor-based systems have been proposed and deployed for a broad range of Categories and Subject Descriptors monitoring (and even actuation) applications. But the vast H.1.1 [Systems and ]: General systems majority of those are targeting a single application or function. theory; I.6.0 [Simulation and Modelling]: General I.6.5 [Model The potential of swarms goes far beyond what has been Development]: Modeling accomplished so far. When realized in full, these technologies will seamlessly integrate the “cyber" (centered today in “the General Terms cloud") with our physical/biological world, effectively blurring Theory, , Verification the gap between the two. We refer to such networked sensors and actuators as the “swarm at the edge of the cloud," and the Keywords emerging global cyber-physical network as the “TerraSwarm," Systems of Systems, Models, Formal Methods. encompassing trillions of sensors and actuators deployed across the . 1. INTRODUCTION To achieve this vision, many fundamental problems have to be In recent years, there has been a frenzy about the potential of solved to prevent potentially catastrophic outcomes of systems interconnecting billions of devices across the entire world. Terms that are so complex to manage. In this respect, IoE, CPS and such the Cyber Physical Systems, Internet of Things, Internet of Swarms need to be considered as evolving systems that are Everything, and Swarm Systems have been the of intense formed of many subsystems that are made to cooperate to reach a research and industrial interest. The market numbers that are goal. The aspects of evolution and cooperation are fully captured circulating are staggering, in the order of trillions of dollars 1 in an older concept: Systems of Systems (SoS). In our opinion, opportunities for the industry. The IBM Smarter Planet initiative systems of the complexity presented above have to be considered is a perfect example of the potential reach of these systems: “At as SoS to make their design and operation feasible. IBM, we want that intelligence to be infused into the systems and processes that make the world work—into things no one would The term System of Systems in its more basic form describes a recognize as computers: cars, appliances, roadways, power grids, collection of components that are themselves systems designed clothes, even natural systems such as agriculture and waterways.” independently and yet in the SoS context are to achieve a common goal. Based on this ‘fractal’ description, we can imagine that a Albeit the original concept of Internet of Things refers to objects component system is itself a SoS until we reach a level of that are connected with a wireless or wired network using all or that one considers the basis of the construction. While part of the Internet protocol (in particular IP), the term is now this is a conceptually interesting view, it is too generic to convey used in a much broader terms where any combination of sensors, the importance of the problems addressed and the potential actuators and computing devices are connected with or without offered by a SoS. in the loop to achieve a goal yielding the concept of Internet of Everything: “Cisco defines the Internet of Everything 1.1 State of the Art (IoE) as bringing together people, process, data, and things to The most well-known modeling work in SoS is currently make networked connections more relevant and valuable than undertaken by the US Department of Defense (DoD) and UK ever before-turning information into actions that create new Ministry of Defense (MOD) through their architectural capabilities, richer experiences, and unprecedented economic frameworks - Department of Defense Framework 2 opportunity for businesses, individuals, and countries” . (DoDAF) and the Ministry of Defense Architecture Framework Focusing on the interaction between the physical and the (MODAF) respectively. These have since been unified in the computing , Cyber Physical Systems (CPS) are also part of OMG 2005 Unified Profile for DoDAF/MODAF [1], and are the recent evolutions. UML profiles specifically intended for capturing most of the important aspects of SoS mainly for military applications. Since then, the SoS domain has gained attention from research and 1 http://www.ibm.com/smarterplanet/us/en/?ca=v_smarterplanet industrial communities, and has today reached a dominant level 2 http://www.cisco.com/web/about/ac79/innov/IoE.html where the potential of SoS to address civil applications can be more work to do. In fact, for us, it is not enough to be able to considered. capture the behavior of SoS with formal languages to reason about In the literature there are numerous definitions of what constitutes properties of SoS and verify whether specifications are met and to a SoS [3-5]. A decade ago, Maier [6], who is considered the operate SoS efficiently when deployed in an evolving “father” of SoS, noted that the term system-of-systems did not environment. Rather a mathematical formalism should be have a clear and accepted definition. However, he acknowledged developed that captures the entire aspects of the definitions that the SoS idea was widespread and generally recognized. He presented in this section. cited a number of examples such as integrated air defense networks, the Internet, intelligent transport systems, and enterprise 1.2 Motivations and contribution information networks, which are an emergent class of systems We argued that the SoS field has suffered from lack of appropriate comprising large-scale systems in their own right. Furthermore, he formalism: most of the definitions and frameworks are based on stated “Systems-of-systems should be distinguished from large but informal considerations and tools are mostly related to monolithic systems by (1) the independence of their components, structure/syntax rather than behavior/. (2) their evolutionary , (3) emergent behaviors, and (4) a To make a difference, we took a different view on SoS that geographic extent that limits the interaction of their components includes the informal characteristics identified in the literature but to information exchange. Even within these properties there are is grounded on a rigorous based on further subdivisions. For example, a distinction between systems mathematical principles so that the following goals can be which are organized and managed to express particular functions, achieved: and those in which desired behaviors must emerge through 1. SoS and their properties are defined in unambiguous terms so voluntary and collaborative interaction.” This led to the that the research and industrial community can exchange ideas characterization of a SoS into five important dimensions, where and results in an easier way than it has been possible until the first of the four characteristics listed above has been split into today; operational and managerial independence [6] [7]. 2. Verification is made possible; 3. Tools can be developed; Additional definitions have been attempted: “There is not one SoS 4. Design flows and methodologies can be established; that is a one- new start, as a single information system 5. Industry can develop products and services that have project. Rather a SoS more typically will be assembled from guaranteed properties and exclude unwanted behaviors. shared reusable components and with many existing systems independently developed for other and various missions.”[9] The paper is organized as follows: We begin in Section 2 with the [10][11]. However, these definitions/characterizations of a SoS description of the conceptual framework using as much as are not without problems. Each definition has its own relevance possible a rigorous formulation. Then we develop in Section 3 the with respect to the application. Also, we note that in the preceding concept of beliefs of the components of the SoS about the definitions there is no direct mention of the requirement for environment, their relationship to and the consequences of collaboration and co-ordination between systems. Interestingly, a inadequate precision including security threats. In Section 4, we more restrictive and quite early definition of a SoS provided by discuss the use of the conceptual model with particular focus on HP comes to the rescue: “Large scale concurrent and distributed coping with failures and we outline future work . systems, the components of which are complex systems themselves (e.g. enterprise networks)” [12]. In essence this refers 2. THE CONCEPTUAL FRAMEWORK to the communications and interactions between the elements of Summarizing the available from the SoS literature the the SoS. The elements are perceived to be complex systems, following characteristics are important [37]: which by communicating with other complex systems form a SoS. 1. Operational independence of the elements: The constituent Also, SoS may overlap with other SoS. This feature becomes a systems can usefully operate independently. sixth major dimension of a SoS. It should be noted that the dividing line between a “system” and a 2. Managerial independence of the elements: The constituent “SoS” is fuzzy, further resulting in confusion within the systems are separately acquired by different managerial community. The above definitions and characterizations of a SoS entities. in their partial view of the problem serve well to illustrate the 3. Evolutionary development: A SoS evolves over time, complex nature of these systems, and suggest that they must be developing its capabilities as the constituent systems are treated very carefully in order to understand and harness the changed, added or removed. benefits. Model-based techniques for the design of SoS have been the 4. Emergent behavior: The SoS itself offers additional services subject of intense research: most relevant are COMPASS above and beyond the capabilities of the constituent systems. (http://www.compass-research.eu/) and DANSE However, it can also exhibit unexpected and potentially th (http://www.danse-ip.eu/home/), two European 7 Framework damaging behaviors. research project devoted to the development of methods and tools for SoS. An excellent survey of model-based techniques for SoS 5. Geographic distribution: The geographic extent of the can be found in [13], supported by COMPASS. Both research constituent systems may be large. consortia have addressed to some extent the use of formal models 6. Coopetition: The constituent systems both cooperate and for designing SoS, (we are part of the DANSE Consortium). The compete with each other in their normal operation. modeling effort (see the extensive list of references listed in [13]) has been focused on “formal” methods where formal is intended For the purpose of this paper, we consider a System of Systems as to underline the use of languages such as SysML, MODAF and a recursively defined entity where the top element (the SoS) is DODAF, and on the definition of interfaces. The tools developed made of a of Constituent Systems (CS). Each CS may be itself in the two projects are useful and should be supported but there is a Systems of System. Our conceptual model covers all these aspects with the addition of safety and security concerns that are which the system was designed to achieve its goals essential when we consider the potential implications of the in a given role widespread use of SoS. 4. A set, possibly empty, of constituent systems, CS. In our conceptual model, a System of Systems is a recursively 5. Capabilities, C, representing what the system can do defined entity where the top element (the SoS) is made of a set of in a given role Constituent Systems (CS). Each CS may be itself a System of • A characterization of all the behaviors the SoS Systems. The leaves of the recursion are atomic CSs that have no can exhibit in a given role. The capabilities of constituent system. The emergency rescue system of Figure 1 has the SoS are totally or partially determined by different constituent systems, such as Fire stations and Hospitals the capabilities of its constituent systems. and Mobile Radio Systems, which by themselves may be • C can be represented by a number of Systems-of-systems, such as the Mobile Radio System. mathematical formalisms such as contract A System of Systems has overarching goals that it attempts to based interface specifications (strong satisfy by corralling the Constituent Systems that have their own assumptions, weak assumptions, guarantees) objectives possibly conflicting among themselves and with the and are often non deterministic. System of Systems´ objectives. A System of Systems has a 6. Behavior, B, the relationship binding the variables potentially limited authority that it exercises on the Constituent (inputs, outputs, states) describing how a system Systems to drive them towards its objectives. operates. The behavior of the SoS is totally or As an example, consider the emergency rescue SoS shown in partially determined by the behavior of its Figure 1. constituent systems and by its own capabilities. A behavior is indeed an instance of the SoS capabilities. 7. A Strategy is the decision of a constituent system about its behavior chosen according to its environment model. 8. Relations, H, defined on the constituent systems and roles of the System of Systems S. o Each relation can be represented as a directed graph. The overall set of relations is represented by a colored directed graph. o We propose the following set of relations in our conceptual model: • COMMUNICATION: a communicates to c, i.e., a can send messages to c), • AUTHORITY: a has authority over c, i.e., a can modify c, for example, by changing its role; typically an SoS exerts some kind of authority on its constituent systems by thus modifying their goals and capabilities, and thus Figure 1 An Emergency Rescue SoS. dictating partially or fully their behaviors; Definition 1 • USE: a uses c, i.e., a can use c by An SoS S = S(R, G, E, CS, B, C, R) is characterized by: utilizing c’s current capabilities to 1. Roles, R, a set of finite roles the system can assume, and satisfy its goals; a finite nondeterministic automata A(R) describing the • OWNERSHIP: a owns c, a can modify c possible role transitions and conditions under which and can grant use to a third party b over these may occur, c; 2. Goals, G, representing for each role what the system • COORDINATION: a coordinates with should do when in a given role: c, i.e., a informs c of its (entire or o A set of objectives given explicitly in terms partial) role, capabilities, and goals; of interface variables of the constituent • OBSERVATION: a observes c, i.e., a systems and parameters and/or implicitly in has access to (can measure) the external terms of inequalities and/or formulae. and/or to the internal variables They are ranked according to priorities. describing the roles, goals, behaviors o Goals can be represented by a number of and capabilities of c; mathematical formalisms such as a • : a knows c, i.e., a prioritized list of LTL formula, equalities observes all variables of c and its and inequalities. They include assumptions, constituent systems. are typically timed and are often Remark probabilistic. The AUTHORITY, USE and OWNERSHIP relations should be 3. Environment Model, E, representing the assumptions partial orders but since the CS may define them locally, for a on the external environment of the system S under particular SoS they need not be partial orders hence yielding an inconsistent SoS. Hence, a structural verification on the relation o pipes (water, gas, oil, …) graph is required for detecting such inconsistencies. o power-lines Definition 2 Classes of constituent systems are parameterized sets o communication networks of constituent systems that share the same structure in terms of o environmental monitoring system roles, environment models, goals, behaviors, capabilities, o traffic monitoring system strategies, and relations. An instance of a class is a constituent o surveillance systems system that differs from other systems in the class only in its parameter values. Definition 5 For example, the class of cars can be defined of which a specific A SoS configuration is determined by model year and make is an instance. o knowledge of all existing classes of constituent The set of Constituent Systems and the set of Relations are highly systems dynamic in the sense that: o knowledge of all existing instances of all existing 1. An element of the set CS may join or leave the set at classes of constituent systems any time; o current relation colored graph between all existing 2. An element joining CS may differ from the present instances of all existing classes of constituent elements in behavior, goal, and capabilities. It can be a systems member of a new class or a new instance of an existing o all the variables of constituent systems that can be class. observed by the SoS. 3. An element joins CS establishing relations with its peers. The relations can be already defined in the Example: Wildfire control System of Systems and in that case, new connections The management of the recent wildfire in Sweden exhibits most may appear or disappear in the directed graph of the dimensions of our conceptual model. In particular it representing the relations, or be new, in that case new highlights the highly dynamic aspects of systems of systems. It links in the graph are added with a different color. also shows how the lack of adequate situation assessments and lack of capabilities caused drastic failures in achieving the A SoS has behaviors that may not be predictable (emerging objectives of the crisis management system in containing the behavior) and that need not satisfy the goals of the SoS because of wildfire and protecting housing. the interactions of its constituent systems that evolve (fully or In the initial phase, a brigade of firemen from Uppsala län was partially) independently. A SoS leverages its authority and the deployed – formally a homogenous system of system with 8 knowledge of the set of CSs to modify and adjust their evolution instances of class firemen, with a simple one-level authority to fulfill its goals. The difficulties a SoS faces are related to partial structure. The available infrastructure was restricted to hoses, with observability and authority as well as the dynamic change of the fire-engines as only of water transportation. The set and of its relations. environmental conditions are characterized by difficult to access terrain, extremely dry forest, and strong winds, with temperatures The problem of verifying the overall dynamics of the SoS, of in the 80°F and sun. determining its strategy towards reaching it goals and of verifying The capabilities of this team were insufficient to contain the fire. the feasibility of reaching its goals given its constituent systems is Within 8 days, the wildfire extended from an initial area of 100 by a very difficult one for its scale that may involve tens of thousands 500 meter to an area covering 150 km2. The SoS went through of leaf CSs and the various types of dynamicity described above. phases of ad-hoc extensions involving help of local farmers in The name of the game is to exploit and leverage its recursive providing additional water transportation facilities, involvement nature by aggregating subsystems in an appropriate way as well as of a fire brigade from a neighboring Västmanland län, still only eliminating unwanted behaviors by construction. In this respect, it with elementary capabilities, provision of a water-bin carrying is convenient to identify two classes of CSs: aggregates and helicopter. Due to the ad-hoc growth, there was no hierarchical infrastructures. authority structure, leading to insufficient coordinated actions. In Definition 3 Aggregates, A, are elements that share goals, illustrating our relations, the helicopter was OWNED by an behavior and capabilities to a point that they can be considered industrial company to the fire-brigades, which then USES this indistinguishable by the SoS either by their own nature or by a resource. While each of the fire-brigade leaders had complete voluntary action of the SoS. An aggregate can be represented by a KNOWLEDGE of his own team members and the employed single element of the CSs at that level of recursion. infrastructure resources, he could only OBSERVE the behavior of the other team due to lack of COORDINATION. Definition 4 Infrastructure, I, is a constituent system that is As the situation escalated, it became necessary to evacuate “passive”, i.e., it does not have authority on other systems, except villages in the area. Different local police brigades where for its own constituent systems, and it does not have intrinsic integrated into the SoS, with the task to secure the evacuation. goals. It is “owned” by a system either at the same level of Their AUTHORITY over the local residents thus allowed recursion or at higher levels in the sense that the owner can changing the roles of local residents, in forcing them to give up determine its goals and behavior as well as grant use of the their current role and evacuate their home. There was, however, infrastructure to other systems including the SoS. Capabilities are no fully established communication relation between the police intrinsic to the infrastructure and are dependent on the capabilities and local residents, leading to inconsistent assessment of the and behaviors of the infrastructure constituent systems. actual state of evacuation, which lead to hazardous situations for Examples of infrastructures are: one family which was not informed and which could only escape o roads the wildfire because of their location at a lake and their own o tracks resources including a boat. Also, due to wrong beliefs about the o airspace future evolution of the wildfire, evacuated persons were bussed to time domain, even in the absence of failures in sensory- a city, which then had to be evacuated itself. and communication systems. Only after escalating the situation and declaring it a national threat • Failures: These effects are aggregated by failures in was a professional management structure established to manage communication and sensory system, which, without the now heterogeneous SoS involving some hundreds of persons proper error-detection and recovery mechanisms can in fire brigades, police, volunteers, and local communities. At this lead to arbitrary large gaps between perceptions of stage, the lack of the previous phase in insufficient capabilities reality and reality itself. and insufficient observability was addressed by adding externally • Compromised information: Finally, all information owned resources, such as surveillance plans with infrared channels (sensory and communication) can be cameras, and four water-bombers provided by Italy and France, compromised, and thus subject to intentional by escalating the containment of the wildfire to the level of the disinformation. European commission. To cater for this in our model, we distinguish between:

The real world W, as perceived by an ideal We will revisit more aspects of this scenario in subsequent o observer with complete information; sections. o Beliefs of constituent systems about the real world, as gained by the constituent system either through 3. REASONING ABOUT SYSTEMS OF direct observations or through communication. SYSTEMS: REALITY, BELIEFS, As pointed out above, such beliefs about the real world, and the PRECISION real world itself may in principle differ arbitrarily (e.g. due to In an ideal situation, all constituent systems would at each point in intruders). We assume the availability of a metric of an time have a correct perception of the real-world. The challenge omniscient observer, which we call precision, for measuring the addressed in this section is the discrepancy between the real-world discrepancy between what A to be true about the real- and what constituent systems believe to be true about the real world, and reality. The construction of such a measure is world. application dependent, but follows principles in so called metric Figure 2 shows an underlying map, with exact locations of the , which, intuitively, measure the degree of falsifying a different instances of constituent system, the location of the crisis formula, with 1 indicating complete satisfaction. E.g in the area, and the spread of a poisonous cloud emanating from the wildfire scenario, the that the forbidden areas are crisis area, constituting a snapshot of the real-world as perceived completely evacuated would have a value inversely proportional by a perfect observer. to the number of humans per unit area in the forbidden zone, There are multiple reasons for discrepancies between the real- being 1 with no humans in the forbidden area, close to 1 with one world and what constituent systems believe to be true about the person visiting the forbidden area, and so on. real world: Similarly, for goals of a SoS, an omniscient observer can measure • Limited direct observability: A constituent system has the degree to which the current SoS configuration satisfies this by itself only limited direct access to information. This goal. E.g. in the wildfire scenario, the goal of eventually contains all information “perceived” through sensory containing the fire within a maximal area a, which can be systems (whether technical or ). E.g. the firemen expressed in first-order temporal logic e.g. by in a building can only perceive the immediate local eventually (fire_area

3https://projects.avacs.org/projects/isat/wiki/ISAT_Quickstart_Gui de. playing against the environment model. used (corresponding to what we called strong assumptions in other contexts). (a1) imprecision can also arise from improper 4. Use of the Conceptual Model: Coping with information management by the involved organizations potentially leading to inconsistent beliefs (calling for a revision of Failures and Future Work potentially all categories of dependency relations, or the need to In this section we demonstrate the value of our conceptual model involve additional organizations having access to more accurate and the capability to reason about precision of beliefs and degrees information). (a2)-imprecision must be addressed by constantly of fulfillment of SoS objectives by giving a classification of monitoring the actual environment and using techniques such as reasons for failing the SoS objectives, and possible lines of attack model-extraction and learning (see e.g. [17]) to maintain to deal with these. For each reason, we address compensating consistency between the actual evolution of the environment and measures. Jointly, these define a to realize a control environment models. Regarding (a3)-imprecision, maintaining strategy for SoS, for compliant to our conceptual coherency between the systems capabilities and its actual health model. In control-theoretic terms, the proposed measures all aim state is essential. at “stabilizing” the system of system in configurations robustly satisfying the objectives of the SoS. While the approach proposed 4.2 Imprecise future state prediction is in general neutral to the concrete modeling used for expressing We propose to base decision making in strategies of constituent the various aspects of our conceptual model, the examples will be systems based on predictions of future evolutions, as in model- drawn from application contexts where models of the real world predictive control. In our setting, each constituent system will use can expressed in first-order logics, capabilities of constituent its currently believed environment- and capability-model to systems can be described by hybrid automata (with modes provide such predictions either through simulation of through corresponding in particular to roles), environment models can be formal analysis, in particular assessing how evolutions are captured as hybrid automata, and objectives can be expressed in potentially leading to hazardous states, or in general threatening to first-order linear time temporal logic. violate objectives of the system of system. E.g. we can (see [18- Figure 3 gives an overview of the possible causes for failing to 22]) perform time-bounded forward reachability analysis for meet the SoS objectives. We now elaborate for some categories (probabilistic) hybrid systems with non-linear dynamics to assess the causes and propose counter-measures, using the wildfire violation of system objectives. Using metric temporal logics scenario for illustrating these. allowing to measure the slackness in satisfying SoS objectives (see e.g. [23]), we can determine the measure of precision to be 4.1 Imprecise Beliefs achieved in our underlying believes of the observations of the real We distinguish three subcategories: our “beliefs” refer to: (a1) world, the environment model, and the capability model. what we observe in the real world (such as the current extent of the fire, (a2) our model of the environment (such as differential 4.3 Poor strategies/Insufficient coordination equations characterizing relevant environmental conditions such We assume a layered formal synthesis approach to determine for as temperature, wind-direction and strength, humidity), and (a3) each layer of the SoS the level of coordination between actors the health state of the constituent system (such as degraded state required to counteract threats to SoS objectives through of helicopters reducing their speed). (a1) imprecision may be coordinated strategies. Formal distributed synthesis approaches deliberately caused by intruders. Confidence may be gained by typically solve the realizability problem for LTL formula, but do applying security protocols (see e.g., [14-16]). E.g. if A has not provide any answers when systems objectives are not received p from B over a secure channel c, and A trusts4 B on the realizable. Reference [24] recently proposed a quality measure basis of a secure authentication protocol, then A can work under for applications such as in CPS, where due to the of rare the design assumption, that trust(c) and trust(B) and received(p, events, realizability is unachievable. Remorse-Free dominant from B, via c) implies p is true in the real world. Or A might work strategies come with the quality attribute that they are better than under the design assumption that his sensor system identifying p any other strategy in achieving systems objectives. This is will never fail, believing that such information will always be particularly relevant for multi-objective systems such as SoS: consistent with the real world. Explicating such assumptions assuming priorities on objectives, remorse-free dominant allows characterizing contexts, in which the system may be safely strategies always achieve all possible priority levels, i.e. give up

4 When referring to high confidence in a security context we use the standard terminology. E.g. a secure channel is one where there is high confidence that the message received over the channel is actually the message send, and trust(A) for saying that the confidence that if A communicates that p is true in his belief system, then the confidence level of p is high. Figure 3 Potential Causes for the Failure of SoS on a lower priority only if this is mandatory for achieving higher- [6] M. W. Maier, 1999 "Architecting principles for systems-of- priority objectives. Hierarchical synthesis approaches have been systems," , vol. 1, pp. 267–284, 9 FEB proposed e.g., in [25-29] for traffic control applications. On a 1999. given abstraction layer, we can determine the degree of [7] A. Sousa-Poza, S. Kovacic, and C. Keating, 2008 "System of cooperation required between constituent systems, as e.g. laid out systems engineering: an emerging multidiscipline," in [30] for the setting of distributed cooperative synthesis of International Journal of System of Systems Engineering, vol. remorse-free dominant strategies. 1, pp. 1-17. 4.4 Human errors [8] A. P. Sage and C. D. Cuppan, , 2001 "On the Systems Engineering and Management of Systems of Systems and How can we assure that beliefs of a human operator on the state of Federations of Systems," Inf. Knowl. Syst. Manag, vol. 2, pp. the SoS are sufficiently precise? How can we organize authority 325-345. in decision making both among human and technical constituent [9] A. J. Krygiel, 1999 Behind the Wizard's Curtain: An systems so that a sufficient level of consistency between their Integration Environment for a System of Systems National beliefs can at all be established under real-time constraints? How Defense University Press, 1999. can we provide sufficient insights not only into beliefs but also [10] Periorellis P. and Dobson J. 2002 , "Organisational Failures strategies to allow cooperative decision making and cooperate in Dependable Collaborative Enterprise Systems," Journal of actions? Integrating humans in real-time decision making in SoS Object Technology, vol. 1, pp. 107-117, 2002. control may suffer from insufficient or poorly designed HMI, [11]Office of the Deputy Under Secretary of Defense for impaired of human operator state such as from stress, fatigue, Acquisition and Technology 2008, Systems and Software overload or health factors or insufficient skills. Human operator Engineering. Systems Engineering Guide for Systems of modeling and human-in-the-loop modeling, such as in [31-34] is Systems, Version 1.0. Washington, DC: ODUSD(A&T)SSE, essential in assessing the real-time capabilities of the human and 2008., Washington, DC. to gain sufficiently precise situational awareness for cooperative [12] V. Kotov, 1997 "Systems of Systems as Communicating decision making. Structures," Hewlett Packard Computer Systems Laboratory 5. Conclusions 1997. The proposed conceptual model of SoS can be used as a blue print [13] C. Nielsen et al., 2014 "Model-based Engineering of Systems for identifying typical trouble spots in SoS design (and of Systems," ACM Comput. Surv., to appear. consequently also for IoT, IoE, networked CPS and swarm [14]. Alvaro A. Cardenas, Saurabh Amin, Shankar Sastry."Secure systems) and guiding the development of distributed strategies Control: Towards Survivable Cyber-Physical Systems" First for the coordination and orchestration of their constituent system. International Workshop on Cyber-Physical Systems Hence, it is of immediate value for the numerous application (WCPS2008). Beijing, China. June 2008. classes discussed in the introduction. Within the space constraints [15] Alvaro A. Cardenas, Tanya Roosta, Gelareh Taban, Shankar of this paper, the level of formalization was chosen carefully to Sastry. Cybersecurity: Basic Defenses and Attack Trends. In convince the reader, that a fully formal definition of both concepts Giorgio Franceschetti and Marina Grossi Eds. Homeland and emergent behaviors as well objectives and robust satisfaction Security. A multi-facet technological challenge. Artech relation is possible. As is apparent from Section 4, the range of House, 2008. methods relevant for designing, controlling and analyzing system [16]S. Amin, G. A. Schwartz, and A. Hussain, In quest of of systems require an open-architecture approach for SoS benchmarking security risks to cyber-physical systems' IEEE Development Environments, which allows the seamless Network Magazine, v.27, n. 1, pp. 19 -- 24, Jan. - Feb., 2013. integration of industry standard methods for SoS modelling with [17]A. Balluchi, L. Benvenuti, M. DiBenedetto and A. the diversity of modelling and analysis methods spanning the Sangiovanni-Vincentelli, The Design of dynamical observers broad number of scientific disciplines demanded to master for hybrid systems: Theory and Application to an Automotive Systems of Systems. Such seamless integration requires an Control Problem, Automatica, Volume 49, Issue 4, April industry wide standardization effort on the relevant concepts, to 2013, Pages 915–925. which our research can contribute. [18]Nils Müllner, Oliver Theel, and Martin Fränzle. Composing thermostatically controlled loads to determine the reliability 6. REFERENCES against blackouts. In Proceedings of the 10th International [1] J. Rice. 2011, The group working on UPDM. (28 October Symposium on Frontiers of Information Systems and Network 2011). Available: http://www.updm.com/ Applications (FINA2014), pages 334 -- 341. IEEE, May 2014. [2] EuropeanCommission, 2009 "Workshop On Systems Of [19]Tao Li, Feng Tan, Qixin Wang, Lei Bu, Jian-Nong Cao, Xue Systems, Contribution To ICT Work Programme 2011," DG Liu, "From Offline toward Real Time: A Hybrid Systems INFSO Unit G3, BrusselsSeptember 21st 2009. Model Checking and CPS Codesign Approach for Medical [3] M. Jamshidi. 2005 System-of-Systems Engineering: A Device Plug-and-Play Collaborations," IEEE Transactions on Definition. Available: Parallel and Distributed Systems, vol. 25, no. 3, pp. 642-652, http://ieeesmc2005.unm.edu/SoSE_Defn.htm March, 2014; [4] M. Janishidi, 2008 "System of Systems - Innovations for 21st [20]Bellemans, T., De Schutter, B., and De Moor, B. (2006). Century," in Industrial and Information Systems, 2008. ICIIS Model predictive control for ramp metering of motorway 2008. IEEE Region 10 and the Third international traffic: A case study. Control Engineering Practice, Conference, Kharagpur 2008. 14(7):757–767; [5] M. Jamshidi, 2008 System of Systems Engineering: [21]Altho, M., Stursberg, O., and Buss, M. (2007). Online Innovations for the 21st Century. Hoboken, New Jersey, USA: verification of cognitive car decisions. In Intelligent Vehicles Wiley, 2008. Symposium, pages 728–733. IEEE, [22]Tichakorn Wongpiromsarn, Ufuk Topcu, and Richard M. Murray, Receding Horizon Temporal Logic Planning, IEEE Transactions on Automatic Control, vol. 57, no. 11, pp. 2817- [31]Jan Charles Lenk, Andreas Lüdtke, Alexandr Puchkovskiy, 2830, Nov 2012. Denis Javaux, Georges Vroonen, Giovanni Scotti, Sonja Sievi, [23]Gerald Sauter, Henning Dierks, Martin Fränzle and Michael Collin Haddow, Felix Flentge, Impact of Improved R. Hansen, Light-weight hybrid model checking facilitating Ergonomics, Collaboration, and HCI in Ground Operations: online prediction of temporal properties, In Danmarks The AERG Study at ESOC, in SpaceOps 2014 Conference, Tekniske Universitet, Editor, Proceedings of the 21st Nordic 2014 Workshop on Programming Theory, 2009 [32]Lüdtke, Andreas, HMI Requirements for Cooperative [24]W. Damm and B. Finkbeiner, Does It Pay to Extend the Advanced Driver Assistance Systems. In: Berliner Perimeter of a World Model? in Proc 17th international Fachtagung für Fahrermodellierung, 2013 Symposium on Formal Methods (FM2011), Lecture Notes in [33]Wenchao Li, Dorsa Sadigh, S. Shankar Sastry, Sanjit Seshia. Computer Science, 2011 Synthesis of Human-in-the-Loop Systems. Tools and [25]H.J. Kim, D.H. Shim and S. Sastry, Flying robots: Modeling, Algorithms for the Construction and Analysis of Systems control and decision making, Intern. Conf. Robotics and (TACAS 2014) Automation, vol. 1, pp. 66-71, 2002. [34]Dorsa Sadigh, Katherine Driggs-Campbell, Alberto Puggelli, [26]B. Horowitz, J. Liebman, C. Ma, T. J. Koo, A. Sangiovanni- Wenchao Li, Victor Shia, Ruzena Bajcsy, Alberto Vincentelli and S. Sastry, Platform-based embedded software Sangiovanni-Vincentelli, S. Shankar Sastry, Sanjit Seshia. design and system integration for autonomous vehicles, Data-Driven Probabilistic Modeling and Verification of Proceedings of the IEEE, vol. 91(1), pp. 198-211, 2003W. Human Driver Behavior. Formal Verification and Modeling [27]W. Damm, H. Peter, J. Rakow, and B. Westphal. Can we in Human-Machine Systems - AAAI Spring Symposium 2014 build it: formal synthesis of control strategies for cooperative [35] Gerald Sauter, Henning Dierks, Martin Fraenzle, and Michael driver assistance systems. Mathematical Structures in R. Hansen, “Lightweight hybrid model checking facilitating Computer Science, 23(4):676{725, 2013; online prediction of temporal properties,” in Proceedings of [28]O. Mickelin, N. Ozay, and R. M. Murray. Synthesis of the 21st Nordic Workshop on Programming Theory, NWPT correct-by-construction control protocols for hybrid systems 09 , Kgs. Lyngby, Denmark, 2009, pp. 20–22. using partial state information. Technical report, [36]W. Damm, G. Pinto, and S. Ratschan, Guaranteed termination http://www.cds.caltech.edu/_murray/papers/mom14-acc.html, in the verification of LTL properties of non-linear robust 2014 discrete time hybrid systems. Int. Journal of Foundations of [29]Javaux, Denis and Wortelen, Bertram and Lüdtke, Andreas Computer Science, 18(1):63{86, 2007. and Pecheur,Charles and Peldszus, Regina and Sievi, Sonja [37]R. Kalawsky, A. Sangiovanni Vincentelli, et al., Challenges and Yushtein, Yuri A Methodology for Analyzing Human- and Approaches to Operationalizing Systems of Systems, Automation Interactions in Flight Operations Using Formal DANSE report, Jan. 2012. Verification Techniques in: Formal Verification and Modeling [38] Lee, E.A.; Rabaey, J.; Hartmann, B.; Kubiatowicz, J.; Pister, in Human-Machine Systems - Papers from the AAAI Spring K.; Sangiovanni-Vincentelli, A.; Seshia, S.A.; Wawrzynek, J.; Symposium, 2014 Wessel, D.; Rosing, T.S.; Blaauw, D.; Dutta, P.; Fu, K.; [30] Werner Damm, Bernd Finkbeiner, Astrid Rakow, What you Guestrin, C.; Taskar, B.; Jafari, R.; Jones, D.; Kumar, V.; always wanted to know about your neighbors, AVACS Mangharam, R.; Pappas, G.J.; Murray, R.M.; Rowe, A., "The Technical Report 105, October 2014, short version submitted Swarm at the Edge of the Cloud," Design & Test, IEEE , for publication. vol.31, no.3, pp.8,20, June 2014