0250-SIP-2017-PIEEE 1

Model and Tool Integration Platforms for Cyber-Physical System Design

Janos Sztipanovits, Fellow, IEEE, Ted Bapty, Xenofon Koutsoukos, Zsolt Lattmann, Sandeep Neema, and Ethan Jackson, Member, IEEE

The tight interaction of physical and computational Abstract—Design methods and tools evolved to support the components creates new generations of smart systems whose principle of "separation of concerns" in order to manage impacts are revolutionary; this is evident today in emerging engineering complexity. Accordingly, most engineering tool suites autonomous vehicles and military platforms, intelligent are vertically integrated but have limited support for integration buildings, smart energy systems, intelligent transportation across disciplinary boundaries. Cyber-Physical Systems (CPS) systems, robots, or smart medical devices. Emerging industrial challenge these established boundaries between disciplines, and thus, the status quo on the tools market. The question is how to platforms such as the Internet of Things (IoT), Industrial create the foundations and technologies for semantically precise Internet (II), Fog Computing and Industrie 4.0 are triggering a model and tool integration that enable reuse of existing “gold rush” towards new markets and accelerating the commercial and open source tools in domain-specific design development of societal-scale systems such as connected flows. In this paper, we describe the lessons learned in the design vehicles, which, in addition to the synergy of computational and implementation of an experimental design automation tool and physical components, involve humans (H-CPS). suite, OpenMETA, for complex CPS in the vehicle domain. The As for most engineered systems, CPS design flows are conceptual foundation for the integration approach is platform- dominantly model-based. The model-based design process based design: OpenMETA is architected by introducing two key results in a formal description (model) of the system S, using platforms, the Model Integration Platform and Tool Integration Platform. The Model Integration Platform includes methods and as input the set of requirements R that must be satisfied in tools for the precise representation of semantic interfaces among some environment E (Fig. 1). modeling domains. The key new components of the Model Integration Platform are Model Integration Languages and the mathematical framework and tool for the compositional R E specification of their semantics. The Tool Integration Platform is Model-based designed for executing highly automated design-space Requirements Design Process Environment exploration. Key components of the platform are tools for constructing design spaces and model composers for analytics workflows. The paper concludes with describing experience and lessons learned by using OpenMETA in drivetrain design and by S System adapting OpenMETA to substantially different CPS application Figure 1: Model-based design domains.

Index Terms — design automation; cyber-physical systems; If S, R and E can be represented formally, the system design model integration; tool integration can be framed as a synthesis process [1] that yields S such that when composed with the environment E, it satisfies the requirements R, or 푆 ∥ 퐸 ⊨ 푅. The automation of the design I. INTRODUCTION process assumes the availability of precise, formal models for YBER-PHYSICAL SYSTEMS (CPS) are engineered systems R and E restricted to a semantic domain where the synthesis C where functionality emerges from the networked process is computationally feasible. However, in CPS, there interaction of computational and physical processes. are significant challenges with deep impact on model-based design flows: This work was supported in part by the Defense Advanced Research Project  Heterogeneity: Heterogeneity is an essential property of Agency under awards #FA-8650-10-C-7075, #FA8650-10-C-7082 # CPS. Much of the advantages of CPS design are expected HR0011-12-C-0008, and by the National Science Foundation under award # CNS-1035655. to come from the synthesis of traditionally isolated J. Sztipanovits, T. Bapty, S. Neema, X. Koutsoukos are with the Institute for modeling domains (creating novel modeling platforms in Software Integrated Systems of Vanderbilt University, TN 37212 USA (e- design flows [2][3]) and from the explicit representation mail: [email protected]). Zs. Lattmann is with A^3 by of interdependences across modeling domains, which is Airbus, CA 95113 USA (e-mail: [email protected]) and E. Jackson is with Microsoft Research, WA 80305 USA (e-mail: frequently neglected if the separation-of-concerns [email protected]). principle [4] is used incorrectly.  Modeling uncertainties: In model-based design R, E and S are models of some referent systems (conceptual, logical 0250-SIP-2017-PIEEE 2

or physical) or their properties. The model is always a emerging from a practical constraint: domain-specific CPS simplification, preserving specifically chosen features of design flows are constructed to support synergistically a the referent. The relationship between models and their design methodology but need to be implemented by referents plays key role in the effectiveness of the design integrating a wide range of heterogeneous tool components. process. In cyber domains, this relationship is usually an This leads to – what Alberto Sangiovanni-Vincentelli and abstraction/refinement relation where property Manfred Broy calls – the “tyranny of tools” in design preservation between an abstract model and its referent automation [9], when not the design problems, but available (e.g. a software implementation) is an achievable goal. In tools dictate the abstractions that designers should use in physical domains this relationship is fundamentally problem solving. different. There is always some level of uncertainty The significance of the problem has long been recognized between a model and its physical referent that may be by industry. In fact the challenge of creating end-to-end aleatoric (irreducible) originating from the underlying integrated domain/product – specific tool chains is considered physics, or epistemic arising from the lack of knowledge to be a primary impediment for the faster penetration of [69]. Measuring, expressing and propagating uncertainties model-based design in CPS industry. Large system companies in CPS design models is an important challenge. face immense pressures to deliver safe and complex systems at  Complexity and scalability: Complexity and scalability of low cost. Tools are at the heart of their engineering process verifying properties of models are significant concerns for covering the full spectrum of requirements, design, all model-based design approaches that target real-life manufacturing and operations support . The internal tool systems. Selecting modeling abstractions with scalable landscapes of large aerospace and automotive companies symbolic or simulation-based verification methods is a contain ~3000-5000 distinct tools totaling several hundreds of key approach in the cyber side of CPS and may result in millions of dollars in internal investments [10]. End-to-end significant complexity reduction [1]. However, on the tooling for these complex CPS product lines spans too many physical side, adjusting the level of abstractions used in technical areas for single tool vendors to fully cover. In modeling without considering the underlying physics addition, a significant part of the companies’ design flow is inevitably leads to increase in epistemic uncertainty that supported by in-house tools that are proprietary and capture may make the verification unsound and/or incomplete. high value design intellectual property (IP). In many areas, This paper is based on our experience with such as drivetrain electronics in the automotive industry, constructing a fully integrated model-based design tool production tool suites include a combination of in-house and chain prototype for the “make” process of complex CPS, commercial off the shelf (COTS) tools in the approximate as part of the Defense Advanced Research Project ratio of 70% and 30%, respectively. The development and use Agency (DARPA) Adaptive Vehicle Make (AVM) of in-house tools is not necessarily the result of deficient program [5]. The resulting design automation tool chain, COTS offerings, but, rather, it is an essential part of the OpenMETA [6], was expected to contribute to the innovation process that yields competitive advantage via following goals of the program: improved product quality and productivity. The primary  Improved design productivity: Shortening technology barrier that slows down this process and makes development times for complex CPS was a primary integration of in-house tools with third party tools extremely goal of the program. OpenMETA contributed to expensive and error prone is the lack of modern tool achieving this goal with the following technical integration and deployment platforms. approaches: (a) extending model-based design to Seamless integration of end-to-end tool chains for highly component - and model - based design, (b) pursuing automated execution of design flows is a complex challenge of correct-by-construction design methods, and (c) which successful examples are rare – even after massive investment by companies. Vendors provide limited executing rapid requirements trade-offs across integration, primarily of their own tools, with a few cross- design domains. vendor integrations for particularly dominant tools (e.g.,  Incorporation of manufacturing related constraints integration with DOORS, Matlab, Word or Excel). This into the design flow to help moving towards foundry- limitation results in design flows that consist of islands of like manufacturing capability for CPS, decreasing integrated tool sub-chains, bridged by various ad-hoc, semi- the need for lengthy and expensive design – automated, or manual stopgaps. These stopgaps impose a manufacture – integrate - test – redesign iterations. variety of costs: additional work in performing manual  Support of crowdsourced design by providing a web- transformations, additional work in guarding against based collaboration platform for ad-hoc, divergence between multiple representations, and forgone geographically distributed design teams and access analysis opportunities, to name just a few. to dominantly open source, cloud deployed tool Truly transformational impact requires an approach for configurations for very low cost. composing an end-to-end integrated tool chain from a The project focused on integrating and testing an heterogeneous collection of COTS, open source, and automated, model-based design flow for the power train and proprietary tools. The ideal solution would support tools from hull of the Fast Adaptable Next-Generation Ground Vehicle multiple vendors, and allow companies themselves to include (FANG) ground vehicle [7][8] using primarily open source the most closely guarded proprietary tools. Such a truly tool components. Given the program focus, our team explored integrated toolset would yield significant improvements in the fundamental challenges of CPS design automation productivity and decreases in design time by eliminating the 0250-SIP-2017-PIEEE 3

unnecessary work associated with the existing integration mechanisms and shortening the learning curves associated with diverse, un-integrated tool suites. In spite of its large significance, integrating design tool Testbench chains today relies on ad hoc methods: engineering process Design Space Configuration are defined, tools are selected covering parts of the process, Space and the emerging isolated islands in the design flow are manually bridged with huge recurring cost, or patched Component Model Testbench Testbench

together with opportunistically constructed data interchange Requirements Model Composition Integration Template mechanisms that cannot be maintained and evolved. Repository Repository Environment The main contributions of this paper are the integration No technologies developed for CPS design automation tool R Multidisciplinary Verification, Testing and E chains. This integration technology is structured around three Optimization horizontal integration platforms: the Model Integration Yes Platform, Tool Integration Platform and Execution Integration S – system model Platform. The primary focus of the paper is the Model and Tool Integration Platforms representing key technology Figure 2: Conceptualization of the design process components of model-based design. These platforms {퐶𝑖(푥, 푝)}, each including a parameter vector x and typed ports incorporate domain agnostic methods and tools for co- p representing the component interface. The component modeling CPS artifacts and engineering processes that can be models are instantiated in specific architectures by setting instantiated in domain-specific contexts. We use the their parameter values. For a system model S, 퐶푆 = OpenMETA design flow that we developed for the FANG 푐표푚푝푡푦푝푒푠(푆) denotes the set of component model types ground vehicle challenge as an example for exposing the instantiated in S (possibly multiple times) and 퐶푆퐼 = integration challenges. We argue that establishing model- and 푐표푚푝푠(푆) yields the set of instantiated component models. tool-integration platforms for CPS design automation helps to The architecture of a system S is defined by a labelled graph create decoupling between domain-specific, and frequently 퐺 , where the vertices are the ports of its components proprietary, engineering processes of systems companies from 푆 and the edges represent interactions corresponding to the type their actual implementation incorporating a large suite of tools of the connected ports. An architecture is well-formed if and extensive IT infrastructure. Since our primary focus is 퐺 satisfies a set of composition constraints Φ over 퐺 derived integration technologies for design automation systems, we do 푆 푆 from the semantics of the interaction types. not intend to provide an exhaustive survey of specific CPS The sets of component types C, and composition constraints design methodologies, but leave it to the excellent papers in Φ define a design space the literature [1][2][3][4] [11][12][23]. 퐷 ≝ {푆|퐺 ⊨ Φ, 푐표푚푝푡푦푝푒푠(푆) ⊆ 퐶} In Section II we provide an overview of the model-based 푆 that includes all possible system architectures that can be design process, the integration architecture in the context of composed from the instances of the parameterized component the OpenMETA design flow and identify the role of the library subject to the composition constraints. The design model- and tool-integration platforms. Section III focuses on process needs to synthesize an 푆 ∈ 퐷 design model for which the model-integration platform and provides details on the 푆 ∥ 퐸 ⊨ 푅. methods and tools used for semantic integration. Section IV As Fig. 2. illustrates, the design process that should yield an focuses on the tool-integration platform. Section V acceptable design model is conceptualized as a design space summarizes the lessons learned and describes current research exploration process that – by using some exploration strategy - directions. incrementally narrows the size of the initial design space until (one or more) satisfying design is found. The core components II. OVERVIEW OF THE MODEL-BASED DESIGN PROCESS of this process are the Multidisciplinary Verification, Testing As described before, a model-based design process receives and Optimization activities that evaluate designs points against requirement models R, environment models E, and synthesizes different requirements in R while operating in targeted a system model S such that ∥ 퐸 ⊨ 푅 . Naturally, system design environments E. Reusing analysis tools and processes is a from “scratch” would not be practical, therefore the structure significant goal, therefore we use modeling and model-based of the design process must allow for reusing as much design integration. From the point of view of the design automation knowledge – models, processes and tools - as possible. There process, these evaluations are performed by testbenches that are two kinds of design knowledge where reuse is critical: incorporate specific configurations of tools for implementing system models and testing/verification methods with related an analysis flow over design models. The goal of the tools. These considerations are reflected in the testbenches is to make a decision if a design satisfies a subset conceptualization of the design process shown in Fig. 2. of the requirements. Testbenches are linked to requirement On the modeling side, model-based design is transformed categories (e.g. mobility requirements that are tested using a into model- and component-based design, where previous dynamic simulation testbench) and well suited for model- design knowledge for the system to be constructed is available based integration. Testbench models that incorporate the in a Component Model Repository. The repository includes a model of an analysis flow and tool interfaces can be set of parameterized component models 퐶 = templatized and placed in a Testbed Template Repository. A 0250-SIP-2017-PIEEE 4 testbed template is instantiated by configuring it with specific tools – that would allow definition of component models, and requirement models, and with a suite of design models design space models, and to populate component and design required by the incorporated analysis tools. The Testbench repositories that can be used/reused by system engineers for Integration process deploys a configured testbench for designing a specific system, b) Model Composition tools – execution. that transform, compose, and derive inputs for domain-specific The last essential condition for improving reuse in the analysis tools from the system design models, c) Domain- design automation process is to enable decoupling between the specific Analysis Testbenches and tools – for analysis and system modeling and analysis sides by introducing a Model evaluation of the candidate system using models of Composition process (see Fig. 2) that composes integrated, progressively deepening fidelities, and d) Analytics and analysis-specific system models from the model of the system Visualization tools – for visualization of analysis results as architecture 퐺푆 and the set of instantiated component models well as for interactively conducting design trades. 퐶푆퐼. We will briefly exemplify the usage of these tools in context of a subset of the FANG vehicle challenge noted above. The simplified view of the tool configuration is shown A. Overview of an Example Tool Architecture: OpenMETA in Fig. 3. The design process described above was prototyped and tested in the OpenMETA tool suite in designing the power 1) Requirements train and the hull of an infantry amphibious vehicle. This In OpenMETA requirements inform the testbenches that are particular challenge was focused on the design of the developed and used to assess the candidate design. The drivetrain and associated mobility subsystems of an infantry drivetrain model of the FANG vehicle had 19 performance vehicle (Fast Adaptable Next-Generation Ground Vehicle requirement types that are divided into five subcategories: FANG). Besides the physical (CAD) and functional (CyPhy / Speed, Acceleration, Range, Temperature/Cooling, and Fuel Modelica) requirements of the drivetrain and mobility Economy. The Speed requirement category contains five subsystems they had to be packaged inside of a predefined maximum speed requirements (forward speed, hill climb on volume in the form of a three dimensional surrogate hull. The different surfaces, and reverse speed) and two average speed designs were evaluated for driving performance on land and requirements. Each requirement has a threshold and an water as well as for manufacturing lead-time and anticipated objective value. The threshold is the pass/fail mark, and the unit manufacturing cost. objective represents the ideal outcome. A score can be Prototyping the conceptual design flow described above assigned to each metric by comparing it to the threshold and required several categories of tools that had to be integrated to objective; exceeding the objective will not necessarily have allow execution of a highly automated workflow. The increased benefit. A design’s overall (weighted) score is integrated tools can be broadly categorized as: a) Authoring computed based on the requirement structure and analysis

Requirements

Architecture Lib. Testbench AnalyticsTempl. Architecture Configuration Exploration Tool Tool Component Lib. Tool Comp. Lib.

Component Design Space Analysis Model Testbench Auth. Tool Auth. Tool Composition Tool Integration Tool multi-models: semantic integration platform analytics-flow models: tool integration platform

Dynamics Mods.

Static Safety Safety Safety Performance FEA FEA Control Testbenches Testbenches Testbenches Testbenches Testbenches Testbenches Testbenches (sim. based (multi-objective (physics-based, Mods. (sim. based) (sim. based) (verification (physics-based, (Visualizer) Dashboard probabilistic) based) optimization) static) dynamic) CAD Mods.

Static Mods. Design-time Seed Design Multi-Model Analytics Testbenches Mods.

Evidence for Performance and Safety Assurance Argumentation: Design-time

Figure 3: OpenMETA Tool Architecture 0250-SIP-2017-PIEEE 5 results, and represents the quality of the design, which in [13]). The (partial) seed design for the vehicle consists of a facilitates the quantitative comparison of different design drivetrain (engine and transmission), left-hand and right-hand solutions. side drive (drive shaft and final drive), cooling system, fuel tank, batteries, and software controllers (Engine Control Unit 2) Component Authoring and Repository and Transmission Control Unit), organized in a topology that The Cyber Physical Component model used in OpenMETA represents the interactions (energy flow, signal flow, etc.) is a novel construct that was designed to address challenges between different subsystems and components. unique to Cyber Physical design. The heterogeneity inherent in Cyber Physical components implied that a singular formalism and model would not be adequate to represent components. In the FANG vehicle design, for example, the Engine was one of the most critical component in terms of its impact on the design requirements. The power curve of the engine (a multi-physics model) models its ability to deliver torque at different engine speeds, and corresponding fuel consumption, and plays a primary role in assessing the gradeability (i.e. whether the vehicle will be able to climb a slope at a desired speed) requirements of the vehicle design. The physical geometry of the engine (a CAD geometry model) is a primary determinant of the 3D placement, manufacturability, and serviceability requirements of the vehicle design. The second challenge was that these different models are typically developed in mature engineering tools (such as Dassault System’s Dymola1 tools, or PTC’s Creo2) with significant investment, and represent the knowledge capital of an organization. The cyber-physical component model had to leverage these artifacts without requiring a redesign in a new formalism. The third challenge was that the component models, as the primary unit of reuse, and their authoring being decoupled from use, Table 1: Engine components in the repository had to be comprehensively characterized, described, packaged, curated, and catalogued according to a taxonomy, such that it In OpenMETA these seed designs can be systematically would allow a systems engineer to later use the components in extended and turned into a design point by introducing a system design. These challenges were addressed by alternative components and parameterization. For example, developing a component specification language, a component the key components of the vehicle design that impact the container format (described in [13][14][15]) to author and Speed and Acceleration requirements are the engine and the package the components, and a component model exchange to transmission components. The seed design is augmented by repository and organize curated components. In OpenMETA replacing the specific engine component (Deutz BFM1015M we leveraged GME, a metaprogrammable modeling (290HP)), with the set of alternative engines shown in Table 1, environment ([16][17]), and customized it with the component and similarly the single transmission component is replaced specification language for authoring components. with set of transmission components from the repository. The Table 1 shows an enumeration of different Engine discrete design space resulting from the single design point components (and their key parameters) authored in the has the same topological structure, however, now encapsulates component specification language (described later) and 200 (25 engines, 8 transmissions) candidate designs. included in the component repository for use in the vehicle design. 4) Design Space Exploration The OpenMETA design flow is implemented as a multi- 3) Design Space Authoring and Repository model composition and testing/verification process that Typically in engineering organizations, seed designs are incrementally shapes and refines the design space using baseline architectures that have been proven through prior formal, manipulable models [14][15]. The model composition usage, and constitute the starting point from which new and refinement process is intertwined with testing and analysis designs are derived by introducing variations in architectural steps to validate and verify requirements and to guide the topologies and innovating in the component technologies. In design process toward the least complex, therefore the least OpenMETA this conceptual process is systematized by risky and least expensive solutions. The design flow follows a formally representing the baseline architectures using a Cyber progressive deepening strategy, starting with early design- Physical Systems Modeling Language (CyPhyML) (described space exploration covering very large design spaces using abstract, lower fidelity models and progressing toward increasingly complex, higher fidelity models and focusing on 1https://www.3ds.com/products-services/catia/products/dymola/latest- release/ rapidly decreasing number of candidate designs. 2 https://www.ptc.com/en/cad/creo 0250-SIP-2017-PIEEE 6

The META design flow proceeds in the following main Multidisciplinary Verification, Testing and Optimization phases: process. 1. Combinatorial design space exploration using static B. Need for Horizontal Integration Platforms finite domain constraints and architecture evaluation (Static Exploration Testbench). Automated execution of the design-space exploration 2. Behavioral design space exploration by progressively process requires the seamless integration of heterogeneous deepening from qualitative discrete behaviors to models and tools. A core technical challenge in creating precisely formulated relational abstractions and to OpenMETA was complementing the traditional, vertically quantitative multi-physics, lumped parameter hybrid structured and isolated model-based tool suites with horizontal dynamics models using both deterministic and integration platforms for models, tools, and executions as probabilistic approaches (simulation-based Dynamics shown in Fig. 4. Testbenches and symbolic Safety Verification Testbenches). The function of the integration platforms are summarized 3. Geometric/Structural Design Space Exploration below. coupled with physics-based non-linear finite element 1. Model Integration Platform. As shown in Fig. 4, a CPS analysis of thermal, mechanical and mobility design-space exploration process cuts across traditionally properties. isolated design domains supported by a wide range of 4. Cyber design space exploration (both HW and SW) tools and valuable model libraries that are accessible in incorporated in dynamics testbenches. popular COTS or open source tools. Many of the languages are complex, not necessarily because of the We gained the following insights with the implementation innate domain complexity, but auxiliary complexities of OpenMETA: caused by an insatiable push for generality and various 1. In a model-based design automation process both incidental tool functions. The CPS design process not component modeling and analysis flow modeling are only utilizes these different modeling languages but it important source of reusability – therefore affordability. need to exploit their interaction expressed as cross- In addition model composition and transformation domain models. Given the semantic complexity of technology is essential for composing integrated analysis integrating and relating heterogeneous models and models in the form required by the various analysis tools. modeling languages, we considered model integration as 2. The complexity of the synthesis process is fundamentally a fundamental functionality that need to be supported by influenced by the size, granularity and richness of the specific methods and tools. The Model Integration model library and the restrictiveness of composition Platform includes languages and tools for defining constraints. In the extreme case, when the component cross—domain model integration languages, formally library includes a single parameterized component that modeling their semantics and the semantics of model can potentially satisfy all the requirements, the design transformations. process is reduced to verifying if a satisfying 2. Tool Integration Platform. Referring to Fig. 3, the parameterization exists and to optimizing the system design-space exploration process incorporates a suite of parameters for some performance metrics by means of the testbenches that implement analysis processes to

Model Integration Platform Analysis Testbench Components Designs Design Spaces Models Models

Composed Analysis/Simulation Models Component Model Cyber Analysis Process Static Geometry Dynamics Verification Exporter Composers Analysis Models Models Composers Tool Integration Platform

Job Manager

Simulation CAD Reliability Verif. SW Open Tools HLA Execution Integration Platform Tools Tools Tools Synth. MDAO

Local File system and/or Cloud Storage Simulation Trace Files, Analysis Results, Computed Metrics

Figure 4. Horizontal Integration Platforms for CPS Design Automation 0250-SIP-2017-PIEEE 7

test/verify satisfaction of requirements. The specification domains is unrealistic. In addition, tool chains that are highly and integration of testbenches is also model-based, configurable to specific application domains are not available. therefore the Tool Integration Platform incorporates both Consequently, our objective with the introduction of the – Model Composers for generating the tool-specific Model Integration Platform was to enable the rapid product models for analysis and Analysis Process construction of domain-specific, end-to-end tool suites for Composers generating executable version of testbench CPS design domains by supporting the modeling of cross- models. The executable testbench models link the tools domain interactions in heterogeneous modeling environments. incorporated in the analysis flow, the corresponding In a naïve approach, the challenge of creating an integrated product models and the integration models the CPS design automation tool chain crossing different design appropriate version of the tool integration framework domains is considered to be a tool interoperability problem used: Open MDAO (Multidisciplinary Design Analysis that can be taken care of with appropriate data conversions and Optimization) optimization tool [18] for parametric and tool APIs. In complex design domains these approaches optimization, or the High Level Architecture (HLA) for inevitably fail due to the rapid loss of control over the distributed simulation [19]. semantic integrity of design flows. The primary reason for this 3. Execution Integration Platform. The automated execution is that the key stakeholders – tool vendors on one side and of analysis processes invokes a variety of tools that need systems companies on the other side – need to respond to to receive and produce models, and need to be scheduled different pressures that drive their product lifecycles. The as the analysis flow requires. The Execution Integration interest of system companies is to control and evolve their Platform incorporates the services and mechanisms product data and engineering processes and use the best-of- required for running the analyses using deployed tools breed tools for implementing their design flows. Tool and generating the analysis artifacts. The first test of the companies are interested in expanding the relevance and OpenMETA tool suite was the FANG-1 challenge prize reusability of their tools across product categories and competition [7][8] focusing on the drivetrain and encapsulating essential parts of end-user design flows. mobility subsystems. Since the competition was also a The role of the Model Integration Platform is to facilitate crowdsourcing experiment with over 1,000 individuals decoupling between these two sides by explicitly modeling grouped in over 200 design teams [8], access to the their relationship. In the model- and component-based design tools and model libraries by the competitors framework of OpenMETA (see Fig. 2 and 3) we structured the across the country was in itself a major challenge. Our Model Integration Platform in the following two layers: Execution Integration Platform (VehicleForge) provided 1. Tool agnostic Model Integration Layer that incorporates web-based delivery platform for the integrated design a Model Integration Language (CyPhyML) for tools, supported their cloud-based deployment through a representing (a) component models, 퐶(푥, 푝), (b) designs: software-as-a service delivery model and incorporated a 푆 = 〈퐶푆퐼, 퐺푆〉, (c) design spaces: D, (d) architecture: 퐺푆 range of collaboration services for design teams. While and composition constraints: Φ, (e) cross-domain the Execution Integration Platform is an important part of interactions, (f) data model interfaces for tools, (g) design automation systems, we do not discuss it in this models of engineering processes and (h) model paper. For further information see [21]. transformations for composing analysis models. The The horizontal model, tool, and execution integration Model Integration Layer is supported by platforms adopt the philosophy and key characteristics of metaprogrammable modeling and model transformation platform-based design [22][23]: tools [16] for rapidly defining and evolving Model 1. Construction of the integration platforms represent Integration Languages and transformations. Details of the distinct, domain independent challenges. Model Integration Layer will be discussed in subsections 2. The integration platforms are not just conceptual but III.A, III.B and III.C. include specific sets of methods, services and tools that 2. The Semantic Integration Layer provides methods and are domain agnostic. tools for formally representing the semantics of the 3. We believe that adoption of generic integration platforms elements of the Model Integration Layer. The primary accelerates the construction of design automation tool reason for introducing formal modeling (metamodeling) chains from reusable components. of the integration languages, data models and model In the following sections we will focus on the Model transformations is that Model Integration Languages are Integration Platform and Tool Integration Platform, present designed to be evolvable: as product categories, our solutions developed first in OpenMETA and use examples engineering processes and tool interfaces are changing, for demonstrating the integration process. the integration models need to be evolved without compromising semantic integrity. Details of the Semantic Integration Layer will be discussed in subsections III.D III. MODEL INTEGRATION PLATFORM and III.E. We believe that the single most important change to achieve The primary benefit of this approach is that Model correct-by-construction design is the introduction and Integration Languages, expressing a tool agnostic view of systematic use of cross-domain modeling. However, creating product architectures and related engineering processes can be design tool chains that cover all potentially relevant CPS designed for simplicity: they do not incorporate all the modeling abstractions and satisfy the needs of all application semantic details present in various domain-specific tools, but focus on cross domain interactions and composition. 0250-SIP-2017-PIEEE 8

In the next parts of this section we provide highlights from These type of considerations led to the detailed design of the Model Integration Language, CyPhyML, that was model integration languages, specifically the Cyber-Physical developed for the FANG design challenge in OpenMETA and Systems Modeling Language (CyPhyML). CyPhyML, is the summarize key features of the selected formal framework composition of several sub-languages as listed below: FORMULA [26][27]. However, we emphasize that the  Component Models (CM) that incorporate (a) several essence of the OpenMETA approach is flexibility; the choices domain models representing various aspects of in the design of CyPhyML were driven by the specific component properties, parameters and behaviors, (b) a set circumstances of the FANG challenge problem. of standard interfaces through which the components can interact, (c) the mapping between the component A. Developing a Model Integration Language: CyPhyML interfaces and the embedded domain models and (d) The CyPhyML model integration language, as noted ealier, constraints expressing cross-domain interactions. was designed to serve the needs of performing automated  Design Models (DM) that describe system architectures design-space exploration for the drivetrain and mobility using assemblies, components and their interconnections subsystems of the FANG vehicle [28] to be followed by a hull via the standard interfaces. design challenge. Consistently with the goals of the  Design Space Models (DSM) that define architectural competition, the design process targeted system-level design, and parametric variabilities of design models as roughly corresponding to conceptual and preliminary design hierarchically layered alternatives for assemblies and phases in the usual systems engineering process, with a path components. toward detailed design. The targeted granularity was on the  Analysis Models (AM) that specify data and control subsystem level including commercial off the shelf (COTS) interfaces for analysis tools. components such as engines. While the design space included  Testbench Models (TM) that (a) specify regions in the embedded controllers, the decomposition hierarchy was design space to be used for computing key performance strongly dominated by physical structure. parameters, (b) define analysis flows for computing key The key considerations in the design of CyPhyML were the performance parameters linked to specific requirements followings: and (c) provide interfaces for visualization. 1. Abstraction layers for integrated design flow 2. Granularity and coverage in the component model library B. Component Modeling in CyPhyML 3. Principles used for constructing the design space Components in CyPhy are the basic units for composing a 4. Scope and depth of the requirements and key design. The component models represent several things about performance parameters the actual component, including its physical representations, For example, evaluation of a drivetrain design for the connections, its dynamic behavior, properties and parameters. mobility requirement ‘Maximum Speed Hill Climb Sand’ To achieve correct-by-construction design, the system models requires a testbench that simulates the lumped parameter are expected to be heterogeneous multi-physics, multi- dynamics of the drivetrain model S composed with appropriate abstraction and multi-fidelity models that also capture cross- terrain data E. For a given drivetrain architecture composed of domain interactions. Accordingly, the component models, in a selected suite of components, the OpenMETA model order to be useful, need to satisfy the following generic composer for lumped parameter dynamics accesses the requirements: dynamics models of the individual components in the 1. Elaborating and adopting established, mathematically architecture and composes them into a system model that can sound principles for compositionality. The semantics of be simulated by a simulation engine (OpenMETA used composition frameworks are strongly different in OpenModelica [29][30] as the primary simulation engine). physical dynamics, geometric structure and computing The component models and the composition mechanism must that needs to precisely defined and integrated. be flexible enough to enable the use of component models of 2. Inclusion of a suite of domain models (e.g., structural different levels of fidelity, (even represented in different models, multi-physics lumped parameter dynamics, modeling languages e.g. Modelica models, Simulink/Stateflow distributed parameter dynamics, manufacturability), on models, FMUs or Bond Graph models). The 푆 ∥ 퐸 different levels of fidelity with explicitly represented composition of the drivetrain model S and the terrain model E cross-domain interactions. is specified in the testbench model by modeling the interaction 3. Specification of component interfaces required for between the terrain and the drivetrain. The testbench links the heterogeneous composition. The interfaces need to be models to the simulation engine to gain an executable for the decoupled from the modeling languages used for evaluation of the ‘Maximum Speed Hill Climb Sand’ capturing the embedded domain models. This decoupling performance parameter. Since all design points in the overall ensures independence from the modeling tools selected design space have the same interface, the testbench model can by the component model developers. be linked to a design space with many alternative, 4. Established bounds for composability expressed in terms parameterized architectures. Using the Open MDAO of operating regimes where the component models (Multidisciplinary Design Analysis and Optimization) remains valid. optimization tool [18], a multi-objective parametric The CPS component model must be defined according to optimization can be performed if the exploration process the needs of the design process that determines (1) the type of requires it. structural and behavioral modeling views required, (2) the type of component interactions to be accounted for and (3) the 0250-SIP-2017-PIEEE 9 type of abstractions that must be utilized during design configured for a specific use. The parameters are largely analytics. We believe that it does not make sense to strive for a similar to properties from a specification standpoint and ‘generic’ CPS component model, rather, component models inherit from a common base class, are associated with a unit, need to be structured to be the simplest that is still sufficient and have a specified default value as well as a validity range. for the goal of “correct-by-construction” design in the given Design space exploration tools use component parameters for context. tuning or optimization of a design, or adaptation of a The CyPhyML Component Model (CCM) was designed to component for a specific use [13][31]. integrate multi-domain, multi-abstraction and multi-language Signal Interfaces are defined by the 푃𝑖푛 set of continuous- structural, behavioral and manufacturing models and to time input signal ports, and the 푃표푢푡 set of continuous-time provide the composition interfaces for the OpenMETA model output signal ports. Interactions via signal interfaces are causal composers consistently with the needs of power train and hull and defined by models of computations [32][33]. Models of design [15]. In Figure 5 we illustrate the overall structure of a dynamics implemented by embedded controllers are CCM for an engine. The main elements of the component represented using causal modeling approaches using modeling model are the port-based component interfaces, the related languages such as Simulink/Stateflow [34], ESMoL [35], embedded domain models and constraints representing cross- Functional Mock-up Units [36] or the Modelica Synchronous domain interactions. Library [37]. Based on the needs of system level integration in FANG, Power Interfaces represent physical interactions using we chose the following component model elements. power flows. With the emphasis on power flows and the Parameter/Property Interfaces include a set of ensuing distinction between physical variables, we follow the parameters/properties ports 푃푝푝 characterizing the components port-Hamiltonian approach to physical system modeling. From using established ontologies. Properties represent component modeling point of view, the approach models the systems as attributes such as material, mass, or an attribute associated energy storing and dissipating components, that are connected with its maturity, e.g. TRL. Parameters are variables that can via ports to power conserving transmissions and conversions. be changed as part of the design process. Properties can be Accordingly, the power interfaces are represented using fixed, or could admit an uncertainty bound with a distribution. acausal modeling framework [38][39][40] to achieve For parameterized components (see parameters below), compositionality in modeling physical dynamics. CyPhyML properties can be variable and algorithmically associated with incorporates the following port types: 푃푟표푡푀푒푐ℎ is a set of

CDieselaterpillar Engine:C9 Diesel E nAVMgine : A ComponentVM Component Param./Property Weight Height Number of Cylinders Maximum RPM Interfaces 680 kg 1070 mm 6 2300 rpm

• characterize Length Width Maximum Power Minimum RPM • configure 1245 mm 894.08 mm 330 kW 600 rpm

Signal Interfaces High-Fidelity Modelica Dynamics Model Rotational Dynamics Signal Port • causal/directional Power Port • logical conn. • no power transfer map Low-Fidelity Modelica Dynamics Model map Rotational Signal Port Power Port Power Interfaces Throttle Power Out Signal • acausal Rotational Bond Graph Dynamics Model Port • physical phen. Power Port Rotational Signal Port (torque/angle..) Power Port Detailed Geometry • power flow Detailed Geometry Model (CAD) Structural Structural Structural Bell Housing Interface Interface Mount Structural Structural Interfaces Interface Interface • named datums FEA-Ready CAD Model map map • surface/axis/point Structural Structural Interface Interface • mapped to CAD FEA Geometry

Figure. 5: Illustration of the Structure of the CyPhyML Component Models parameters. Properties can also be scalar or vector, and have a rotational mechanical power ports, 푃푡푟푎푛푠푀푒푐ℎ is a set of unit associated with them. CyPhy employs a standard unit translational mechanical power ports, 푃푚푢푙푡𝑖푏표푑푦 is a set of system based on NASA’s QUDT unit ontology. Figure 5 multi-body mechanical power ports, 푃ℎ푦푑푟푎푢푙𝑖푐 is a set of shows examples for the properties characterizing a Diesel hydraulic power ports, 푃푡ℎ푒푟푚푎푙 is a set of thermal power Engine. Parameters are mechanism to capture component ports, and 푃푒푙푒푐푡푟𝑖푐 is a set of electrical power ports. Each variability, allowing customization for a specific instantiation power ports is associated with two physical variables (effort or use. For example, a variable-length drive-shaft component and flow), whose product yields power. In this approach, can be represented with a length parameter that can be dynamics models are represented as continuous time 0250-SIP-2017-PIEEE 10

Differential Algebraic Equations (DAE) or hybrid DAEs. mismatch is created between the CyPhyML Component Since model libraries may come from different sources, Model and the embedded domain models. component models are potentially expressed in different To solve this problem, OpenMETA includes a full tool suite modeling languages such as Modelica [37] or Bond Graphs for importing domain models (such as Modelica dynamic Error! Reference source not found.(although we dominantly models), integrating them with standard CyPhyML used Modelica-based representations). The use of multi- Component Model interfaces, automatically checking fidelity models is important in assuring scalability in virtual compliance with the standard, and automatically checking prototyping of systems with large number of complex model properties, such as restrictions on the types of domain components. models, well-formedness rules, executability, and others. Structural Interfaces provide interaction points to the Based on our direct experience, the automated model curation geometric structure of components usually expressed as CAD process resulted in orders-of-magnitude reduction in required models of different fidelity. Geometry is a fundamental aspect user effort for building Component Model libraries. of CPS design. The structural interface incorporates geometric In summary, CPS component models are containers of a ports 푃𝑔푒표푚 , where component geometries can be connected selected set of domain models capturing those aspects of and provide the geometric constraints from which the component structure and behavior that are essential for the composed geometry of an assembly is calculated. They are the design process. While the selected modeling domains are basis for deriving geometric features of larger assemblies and dependent on CPS system categories and design goals, the performing detailed Finite Element Analysis for a range of overall integration platform can still be generic and physical behaviors (thermal, fluid, hydraulics, vibration, customizable to a wide range of CPS. electromagnetic and others [15].) C. Design and Design Space Modeling Domain Models capture model artifacts specific to supported domain tools. The Domain models in CyPhyML are The 푆 = 〈퐶푆퐼, 퐺푆〉 generic design model is refined in a wrapping construct that refers to the domain model in its CyPhyML into the tuple native representation language and tool. The Domain model exposes the core interface concepts that are used to map into 푆 = 〈퐶푆퐼, 퐴, 퐹, 푃, 푐표푛푡푎푖푛, 푝표푟푡푂푓, 퐸푃, 퐸푆, 퐸퐺, 퐸푉〉 the Component Interfaces. For example, the component power interface map directly into Modelica power ports. The with the following interpretation: supported set of domain models are a point of expansion for - 퐶푆퐼 is a set of component instances the language. In many OpenMETA transition projects, several - 퐴 is a set of component assemblies, new Domain Models have been added. For the FANG-1 - 퐹 is a set of formulas competition, supported domain models included (a) Modelica - 퐵푒 = 퐶푆퐼 ∪ 퐴 ∪ 퐹 is the set of design elements, models, Bond Graph models, Simulink/Stateflow models, - 푃 is the union of ports included in the component STEP compliant CAD models, and restricted Manufacturing interfaces, ∗ models representing cost, lead time and process information. - 푐표푛푡푎푖푛: 퐵푒 → 퐴 is a containment function, whose range CCM allows multiple domain models for each domain thereby is 퐴∗ = 퐴 ∪ {푟표표푡}, the set of design elements extended enabling a multi-fidelity representation of the component. In with a special root element root, case there are multiple domain models, they are tagged with - 푝표푟푡푂푓: 푃 → 퐵푒 is a port containment function, which fidelity tags. The fidelity tags are kept freeform to allow users, uniquely determines the container of any port, component modelers, and tool developer’s flexibility in - 퐸푃 ⊆ 푃푃 × 푃푃 is the set of power flow connections specification and usage, since there is currently no universally between power ports, accepted taxonomy of model fidelity. - 퐸푆 ⊆ 푃푆 × 푃푆 is the set of information flow connections Formulas are modeling constructs for specifying between signal ports. parametric interdependences across domain models. They are - 퐸퐺 ⊆ 푃퐺 × 푃퐺 is the set of geometric structure used in conjunction with ValueFlow that establishes the connections between structure ports. dependency relation while formulas define the mathematical - 퐸푉 ⊆ 푃푃푃 × 푃푃푃 is the set of value flow connections transformation between parameters. They are used extensively between parameter and property ports. as a method for cross-domain modeling. The restrictions over composing models by allowing the The construction of CCM component models from a library formation of only four type of flows 퐸푃, 퐸푆, 퐸퐺, 퐸푉 represent of domain models, (such as from Modelica models the composition constraints Φ defined earlier. Even with these representing lumped parameter dynamics of physical or constraints, if we adopted the design space as the computational behaviors, CAD models, models of properties 퐷 ≝ {푆|퐺푆 ⊨ Φ, 푐표푚푝푡푦푝푒푠(푆) ⊆ 퐶} and parameters and cross-domain interactions and the set of all possible combination of components via well-formed mapping of domain modeling elements to component flows, we would get design space that incorporates almost interfaces), are time-consuming and error prone. The most exclusively useless design points. To restrict the design space, important challenge is that the CyPhyML Component Model we used a common heuristics that starts all new designs from defines interface semantics (e.g. acausal power interfaces) that modifying an old one. Accordingly, we provided for designers represent restrictions over the modeling languages used in the seed designs 푆푠, that are functional, fully instantiated and behavior and structural models embedded in components. tested architectures together with a set of operations that they These restrictions need to be enforced otherwise a semantic could use to expand the 푆푠 design points into a design space 0250-SIP-2017-PIEEE 11

Semantics of the D. Formal Framework for Semantic Integration: FORMULA integration language The “cost” of introducing a dynamic model integration language is that mathematically precise formal semantics for

Semantic Interface Semantic Interface model integration had to be developed. Because no single tool uniformly represents all aspects of a CPS system, a holistic CPS engineering process must include a method to define, Model Integration compose, store, and evolve models in a tool-independent Domain Modeling Language Domain Modeling Language (signal) Language (physical) fashion. The only truly unambiguous and tool-independent method is a mathematical one, which defines: (1) a standard representation for models that is suitable for many domains, Domain Modeling Domain Modeling (2) a formalism for separating valid models from invalid ones Language (prop/par) Language (geom. with an acceptable degree of precision, and (3) a class of functions for transforming one model into another. These definitions provide a semantic backplane supporting CPS Semantic Interface Semantic Interface model composition and evolution throughout the engineering Figure. 6: Semantic interface between the integrated lifecycle. languages and the model integration language Armed with this semantic backplane, tools may be implemented that automate its application. For example, a tool by introducing structural alternatives (alternative, compatible might be implemented that efficiently checks if a CPS model components and assemblies) and parametric ranges. While this is valid (e.g. does not violate certain cross-domain electro- approach rapidly converges to new designs, it certainly biases mechanical constraints) according to unambiguously-stated initial thinking in some direction. We believe that design formal validation rules. One implementation may be more (or space construction is a hugely important area of design less) efficient for specific operations, such as the automation that deserves significant attention in the future. transformation of large existing models or the synthesis of From the point of view of the design of the CyPhyML new models from scratch. Most importantly though, models, model integration language, the central issue is how the model validation, and model transformations remain as formal selected component modeling and design modeling concept concise tool-independent objects. impacts the semantics of the integration language. It is For the remainder of this section we describe our important to note that we do not intend here to extend the experiences developing a semantic backplane for CPS, which considerations to the details of modeling languages used for led to the implementation of the FORMULA 2.0 system. CPS, we leave this to the many excellent articles [1]-[4] Before discussing specific design decisions, it is important to [12][33][41]. emphasize a “no silver bullet” principle for semantics The relationship between the semantics of the domain backplanes: A semantic backplane cannot capture the formal modeling languages and the model integration language is semantics of all modeling domains with perfect fidelity while illustrated in Fig. 6. As the figure shows, the model integration simultaneously having practical implementations. For language provides a precisely controlled level of decoupling example, consider a semantic backplane that could validate between the frequently vastly complicated domain modeling vehicle models for complex errors in their physical dynamics. languages (such as Modelica for lumped parameter dynamics Any implementation of such a backplane most likely or STEP/IGES for CAD) using semantic interfaces. The subsumes the functionality of Matlab or Modelica, which primary goal for their design is simplicity: select semantics already have hugely complex implementations. Also, consider that is sufficient for the integration of design models and that a vehicle likely has embedded software written in a C-like engineering process models. To preserve flexibility without language. A semantic backplane that could detect hard-to-find losing precision, the semantics of the model integration bugs in this code via a perfect understanding of C semantics language and that of the semantic interfaces need to be most likely subsumes a C compiler, which again is another explicitly defined. These semantic specifications are collected complex implementation. There is no silver bullet and no in the Semantic Backplane in OpenMETA. There are two utopian formalism that has perfect formal semantics while kinds of semantic specifications required: being practically implementable. Instead, a pragmatic 1. Structural semantics (or static semantics) [24] that approach needs to be taken. The backplane focuses on specifies constraints that all model instances of the distinguishing models that are likely to be valid with an language need to satisfy. In the case of CyPhyML, this is emphasis on cross-domain validation rules. It also supports particularly important, since the composition rules for model transformations that can create detailed projections of creating models from components have many type complete system models, and these can be further analyzed by restrictions for obtaining valid power flows and geometric domain-specific tools (e.g. ODE solvers, finite-element structure. A more detailed description is available in [42]. analysis tools, or software verification tools) without 2. Behavioral semantics for the composition operators that reinventing the entire wheel. connect interface ports. Several examples for the A semantic backplane must provide a convenient formalism specification of denotational semantics for composing for representing, validating, and transforming models. Each of power flows as constraints over effort and flow variable these goals implies a non-trivial set of design decisions that pairs (Kirchoff laws) are presented in [43][44]. balance convenience, expressiveness, and efficiency. In fact,

0250-SIP-2017-PIEEE 12 many of the design decisions are related to ongoing debates in 1: domain Deployments many fields of computer science, 2: { 3: Service ::= new (name: String).  Model representation: CPS models take the form of 4: Node ::= new (id: Natural). graph-like structures (e.g. Matlab / Simulink), code-like 5: Conflict ::= new (s1: Service, s2: Service). structures (e.g. systems ODEs & C code), and relational 6: Deploy ::= fun (s: Service => n: Node). structures (e.g. maps of model parameters to parameter 7: 8: conforms no { n | Deploy(s, n), Deploy(s', n), values). Should these all be reduced to relations over 9: Conflict(s, s') }. atomic data types (e.g. Alloy, logic programs without 10:} function symbols, relational databases)? Should they all be expressed as trees of data (e.g. JSON, XML, Coq)? The Deployments domain formalizes the following Should they be some mixture of the two (NoSQL, cross-domain problem: There are services, which can be in document databases, graph databases)? Decisions at this conflict, and nodes, which can run services. Services must be level impose classic burdens on the comprehensibility of deployed to nodes such that no node executes conflicting models. services. Lines 3 – 6 introduce data types to represent the  Model validation: How should users express rules which entities of the abstraction. The conformance rule (lines 8-9) separate (likely) valid from invalid models? Classic forbids conflicting tasks to run on the same node. This is an examples are first order logic, graph constraints, (finite-) example of an OLP rule, which can be interpreted either as a automata, and functional and procedural programs. Again, logical axiom on valid models, or as a program that searches tradeoffs of expressiveness, comprehensibility, and over a model checking for violations of this rule. Constructing efficiency occur here. a valid model for a fixed set of tasks, conflicts, and nodes is  Model transformation: How should functions be NP-complete. It is equivalent to coloring the conflict graph formalized that transform one model to another? Classic with nodes, demonstrating that model construction can be examples are string and tree transducers, term rewrite difficult for humans and machines. systems, functional and procedural languages, and text Domain Composition. FORMULA 2.0 provides templates. Should this be a distinct formalism from model composition of domains so abstractions can be composed. For validation? example, Deployments can be constructed by gluing The approach we have taken represents models as sets of together two independent domains, and then adding additional tree-like data (similar to modern NoSQL databases) allowing a validation rules and data types. Here is an example of a uniform encoding of tree-like, graph-like, and relational refactoring Deployments into three separate domains. structures. We have unified model validation and 1: domain Deployments extends Services, Nodes transformation using open-world logic programming (OLP), 2: { which allows both validation and transformation operations to 3: Conflict ::= new (s1: Service, s2: Service). 4: Deploy ::= fun (s: Service => n: Node). be formalized as an extension of first-order logic with fix- 5: point operations. Axioms written in this logic can be dually 6: conforms no { n | Deploy(s, n), Deploy(s', n), understood as executable programs, providing engineers with 7: Conflict(s, s') }. an additional mental-model for comprehending their 8:} specifications. Our specifications are highly declarative and easily express The domains Services and Nodes contain the data complex domain constraints, rich synthesis problems, and types and validation rules for a valid set of services and nodes. complex transformations. Automated reasoning is enabled by This composition semantically merges data types (or produces efficient symbolic execution of logic programs into quantifier- an error) and conjoins inherited constraints with new free sub-problems, which are dispatched to the state-of-the-art constraints. In this way, large complex domains can be built SMT solver Z3 [45]. FORMULA 2.0 has been applied within up from smaller pieces. The FORMULA 2.0 system provides Microsoft to develop modeling languages for verifiable device a formal meaning for this composition and its implementation drivers and protocols [46]. It has been used by the automotive checks for many logical inconsistencies automatically, embedded systems industries for engineering domain-specific including semantically conflicting definitions of data types, languages [47] and design-space exploration [31] under hard un-satisfiable rules, and rules that produce badly-typed resource allocation constraints. It is has been used to develop inferences. semantic specifications for complex cyber-physical systems. Models. Models are represented simply as sets of well- FORMULA 2.0 is released under an open-source license typed trees created from domain data types. Model modules and can be found at https://github.com/Microsoft/formula. hold the set of trees. Note that by allowing for a set of trees, it Domains. The structure of models and validation rules for is possible to naturally represent the full spectrum of models models are specified using algebraic data types (ADTs) and from purely relational, to graph like, to fully tree-like, and OLPs, as shown in the example below. FORMULA 2.0 rules interact easily with this spectrum of representations. Formally, a domain D is an OLP. A model M closes D with a set of facts, written D[M]. A fact is just a simple rule stating that a data element is provable, and so a model is simultaneously a set of facts. The properties of a model M are those properties provable by the closed logic program D[M]. 0250-SIP-2017-PIEEE 13

In the example below, a model of an automotive system has been successfully applied to perform aspects of design- contains two services, one for voice recognition and one for space exploration. handling the car’s dashboard user interface. Due to Transformations. Transforms are OLPs that transform computational constraints, these two services cannot be placed models between domains. They are useful for formalizing on the same compute node. changes in abstractions (such as compilers) and for projecting 1: model Undeployed of Deployments large integrated models into consistent submodels that can be 2: { fed to domain-specific tools. Below is a simple example that 3: sVoice is Service("In-car voice recognition"). 4: sDB is Service("Dashboard UI"). compiles Deployment models into configuration files. 5: n0 is Node(0). 1: transform Compile (in::Deployments) 6: n1 is Node(1). 2: returns (out::NodeConfigs) 7: Conflict(sVoice, sDB). 3: { 8: } 4: out.Config(n.id, list) :- 9: model Good of Deployments extends Undeployed 5: n is in.Node, 10: { 6: list = toList(out.#Services, NIL, 11: Deploy(sVoice, n0). 7: { s.name | in.Deploy(s, n) }). 12: Deploy(sDB, n1). 8: } 13: } 9: domain NodeConfigs 14: model Bad of Deployments extends Undeployed 10: { 15: { 11: Config ::= 16: Deploy(sVoice, n0). 12: fun (loc: Natural -> 17: Deploy(sDB, n0). 13: list: any Services + { NIL }). 18: } 14: Services ::= 15: new (name: String, 16: tail: any Services + { NIL }). Model composition is like domain composition, and allows 17: } sets of trees to be combined. Each of these related models differs in the properties that can be derived from their contents Models of the NodeConfigs domain contain node (i.e. closed logic program). configuration files (lines 11-16). Each file lists the services that run on a node. These are modeled using recursive ADTs  Deployments [Undeployed] does not satisfy (i.e. utilizing the full power of tree-like representations). The conforms, because services are not deployed to nodes. Compile transform takes a Deployments model called in (Violates validation Rule in Line 6 of Deployments and produces a NodeConfigs model called out. This is domain, which requires an instance of the Deploy data accomplished by the rule in lines 4-7. This rule converts every type for every instance of Service data type.) node into a configuration file containing a list of services. It  Deployments [Good] satisfies conforms, because all employs the built-in toList function which joins a set of services are deployed and all conflicts are respected. small trees into on large tree (in this case in the form of a list).  Deployments [Bad] does not satisfy conforms, because its Notice that models, domains, and transforms all use the same deployments violate conflicts data types and logical rules the describe their behavior. They can all be composed. In fact, the Compile transform has a Partial Models. Partial models partially close domains. A copy of Deployment and NodeConfigs domains within partial model P is solved by a model M if all facts contained it, and its rule can refer to data types and have access to within P and all requires clauses of P are provable in D[M]. In derivations performed by rules in these domains. this way, partial models describe many problem instances, and solving a model is equivalent to synthesis. The partial model E. OpenMETA Semantic Backplane below describes a specific deployment problem, and there are The second component of the Semantic Integration Layer, is infinite set of models that solve it. 1: partial model SpecificProblem of Deployments Epistemic Semantics 2: { (ontologies: shared conceptualization) 3: requires Deployments.conforms. 4: Ontologies Units 5: sVoice is Service("In-car voice recognition "). Architecture Requirements System Models 6: sDB is Service("Dashboard UI"). Testing 7: n0 is Node(0). … 8: n1 is Node(1). 9: Conflict(sVoice, sDB). 10: } The assertions in lines 3-9 must hold in a solution. Requires DSML Semantics clauses state more complex requirements on solutions, e.g. (structural) line 3 requires models to conform to the Deployments domain. (behavioral – denotational) The Good model is a manually constructed solution to this (behavioral – operational) partial model. However, with modern constraint solver, such Load Aerodynamics as Z3, it is sometimes tractable to automatically synthesize NASTRAN FEA complete models. Tractability depends on the size of the Figure 7: Dimensions of Semantics partial model, structure of constraints, and degrees of freedom the OpenMETA Semantic Backplane. Its primary role is the that need to be resolved. In the AVM program such synthesis 0250-SIP-2017-PIEEE 14 collection of all model and tool integration related semantic specifications as formal models of all key aspects of the In summary, the tool agnostic Model Integration Layer that domain-specific integration solution. incorporates a Model Integration Language (CyPhyML) (with As shown in Fig. 7, there are two semantic dimensions that sublanguages for representing component models, designs, need to be considered in a domain: design spaces, cross-domain interactions, composition 1. Epistemic Semantics that capture shared constraints, data model interfaces for tools, models of conceptualization using ontologies. These ontologies engineering processes and model transformations for incorporate standards such as QUDT3 ontology for units composing analysis models) is complemented by the Semantic of measure, quantity kind, dimensions and data types Integration Layer including the FORMULA-2 based Semantic originally developed for the NASA Exploration Initiatives Backplane. Since the Model Integration Language is designed Ontology Models (NExIOM) project, and many domain, for evolution, defining formal, mathematical semantics of its even company-specific ontologies defined for improving components is essential for keeping tight control over the interoperability of engineering processes. semantic integrity of the design process. 2. Formal metamodels (models of modeling languages [17]) of Domain Specific Modeling Languages (DSMLs) IV. TOOL INTEGRATION PLATFORM defined for representing integration models such as the The OpenMETA Tool Integration Platform (see Fig. 4) Model Integration Language (in the example CyPhyML) comprises a network of Composers implemented as model and its sublanguages, Semantic Interfaces to domain transformations that compose models for individual tools (e.g. models, and modeling languages specifying design flows Modelica models from CyPhyML design models and in testbenches. Formal metamodeling in FORMULA-2 component models) invoked by testbenches and deploy also enables the specification of model transformations in model-based design flows on standard execution platforms Model Composers extensively used in the Tool such as MDAO or HLA. Model-transformations are used in Integration Platform (see IV.A). the following roles: 1. Packaging. Models are translated into a different These two dimensions of semantics intersect such that syntactic form without changing their semantics. For (ideally) the various DSMLs created for supporting model example, AVM Component Models and AVM Design integration utilize accepted ontologies – standards or locally Models are translated into standard Design Data defined. The OpenMETA project followed this principle as Packages for consumption by a variety of design much as it was practical given the project time frame. analysis, manufacturability analysis and repository tools. The Semantic Backplane is a collection of all formal 2. Composition. Model- and component-based technologies metamodels and specification of model transformations are based on composing different design artifacts (such as accessible via a web interface to inspect, update and execute DAE-s for representing lumped parameter dynamics as them on the FORMULA-2 tool. The Semantic Backplane was Modelica equations [37], input models for verification close to 20,000 lines of FORMULA-2 code, of which about tools [48]-[54], CAD models of component assemblies 60% was autogenerated from MetaGME – based graphical [15], design space models [55][61], and many others) metamodels [16]. Examples and more details of the formal from appropriate models of components and component metamodels for OpenMETA are available in several papers architectures. [14][42]-[44] . 3. Virtual prototyping. Several test and verification methods We believe that the introduction of Semantic Backplane is a (such as Probabilistic Certificate of Correctness – PCC) new approach to constructing complex component- and require testbenches that embed a virtual prototype of the model-based design tool chains. It is an essential tool for those designed system executing a mission scenario in some who design and evolve domain specific tool chains and environment (as defined in the requirement documents). responsible for the overall integrity of the model and tool We found distributed, multi-model simulation platforms configurations used in the design process. Its importance was the most scalable solution for these tests. We selected the proven in the following use cases: High Level Architecture (HLA) as the distributed 1. As in all areas of engineering, mathematical modeling simulation platform and integrated FMI Co-Simulation helped designing and evolving modeling languages, components with HLA [19]. composition semantics and model transformations. It was 4. Analysis flow. Parametric explorations of designs (PET), invaluable in finding and correcting inconsistencies, such as analyzing effects of structural parameters (e.g. identifying incompleteness problems, and fixing errors in length of vehicle) on vehicle performance, or deriving the semantic foundations of the tool chain. PCC for performance properties frequently require 2. The FORMULA-2 based executable specifications of complex analysis flows that include a number of model transformations were used for generating reference intermediate stages. Automating design space traces and served as abstract prototypes for constraint explorations require that Python files controlling the checkers and production-level transformations used execution of these flows on the Multidisciplinary Design throughout the tool chain. Analysis and Optimization (OpenMDAO ) platform (that 3. The CyPhyML Reference Manual was auto-generated we currently use in OpenMETA) are autogenerated from from the formal specifications. the testbench and parametric exploration models (Fig. 4).

3 http://www.qudt.org/ 0250-SIP-2017-PIEEE 15

A. Model Composers configurations the Model Composers extract the configuration The Model Composers are the bridge between the Model, and view specific content and perform the model Tool, and Execution Integration Platforms by (1) composing transformation to simulation, analysis, or verification tools. testbench (and incorporated tool) specific analysis models from designs models 푆퐷 = 〈퐶푆퐼, 퐺푆〉, 푆퐷 ∈ 퐷 that are included B. Example Model Composer for PCC Testbench in the D design space using the information in the component As an example for a Model Composer we discuss the models, 퐶(푥, 푝), (2) integrate those with the testbench models Probabilistic Certificate of Correctness (PCC) testbench and to obtain executable specification and (3) map the fully its PCC Model Composer, which generates a PCC experiment configured testbench models on the Execution Integration setup for the Execution Integration Platform. Each testbench is Platform [55]. applicable for design spaces, thus the PCC experiment is Model Composers work on a specific view of models and applicable over a set of configurations generated by the design implement a logic that relies on the structural semantics of a space exploration tool. The purpose of these experiments is to domain or domains associated with the specific view. The find the most robust design w.r.t. changes and uncertainties in Model Composers extract relevant portions of the model from the input parameters to the workflow. A distribution function a selected view. Once the specific view is selected a model is fitted on each output. Each output has three parameters transformation is performed to another model representation, defined: a minimum value, a maximum value, and a target while the semantic specifications are preserved. The percentage. The area under the output distribution function transformed models are executable or analyzable by within the defined minimum and maximum values gives the simulation, static analysis, or verification tools. PCC value, a number between 0% and 100%. If this PCC There are several approaches to implement Model value is higher than the defined target PCC value in the model, Composers: it means that the output parameter falls within our required 1. Using manually implemented and maintained source limit. Often this target PCC value is directly linked to a system code in a programming language. or subsystem requirement. For each design a joint PCC value 2. Using a combination of manual implementation of is derived from all outputs. source code in conjunction of automatically PCC testbench model contains a definition of a workflow of generated Application Programming Interfaces analysis tools and a PCC driver module model as depicted in (APIs) for the structural semantics of each model Fig. 8. The workflow model elements refer to other representation. testbenches, which are using the Execution Integration 3. Using model-based rewrite rules, where no or Platform to generate analysis results. A dependency between minimal logic is captured by manually implemented the different testbenches, most likely spanning vastly different source code [16]. domains, are contained in the PCC testbench model. The PCC 4. Using a formal specification of the Model Composer driver model defines a set of input parameters for the (e.g., FORMULA). workflow and their probability distribution functions (PDF) All implementation approaches starting from 2 have and a set of instrumentation points, which variables are progressively stronger linkage and relationship to the semantic recorded throughout the multiple execution steps. In addition, backplane. This provides significant benefit on verification of the PCC driver model defines a sampling method for the input ensuring semantic consistency across the multiple parametric space, which determines the number of iteration for representations and guarantee that all concepts are kept this experiment. consistent. However, our experience shows that as we move towards more formal specification of Model Composers, the Model Executable model with uses runtime of Model Composers gets larger. Model Composers Testbench 1 Composer 1 Analysis Tool 1 observe scalability issues, when dealing with large scale PCC Model Composer Integrated experiment complex CPS models along with rich domain specific Executable model with uses Testbench 2 Model setup in languages containing thousands of concepts. Composer 2 Analysis Tool 2 OpenMDAO These Model Composers also work on design spaces and on the instantiated testbenches from the testbench template PCC Model Composer PCC Model PCC driver as an PCC driver Composer OpenMDAO driver

PDF inputs Record distributions PCC driver outputs Figure 9: An example for Model Composer (PCC)

A PCC Model Composer was implemented to map the PCC Testbench 1 Testbench 2 testbenches to the Execution Integration Platform. Fig. 9 pass parameters Model Composer 1 Model Composer 2 depicts the logic of the PCC Model Composer. Each Model Analysis Tool 1 Analysis Tool 2 Composer is reusable, functionally complete and composable. For example, the Model Composer 1 is reused as-is for Figure 8: PCC testbench model generating executable models for Analysis Tool 1. We utilize library. This capability provides a high degree of automation, the capabilities of OpenMDAO Error! Reference source not when a design space is elaborated to hundreds or thousands of found. to perform the execution. The PCC Model Composer invokes all required Model Composers and generates 0250-SIP-2017-PIEEE 16 additional OpenMDAO component wrappers for each structures. It is often tempting to trivialize the type of low- testbench type around the executable models. In addition to fidelity analytics employed in static DSE and their role in the OpenMDAO components, the PCC driver model is understanding complex CPS designs. However, we learned transformed to an OpenMDAO driver. The driver module is a that several common classes of design errors (“sizing parametric OpenMDAO driver developed by Oregon State mismatch”, “part compatibility”, “manufacturability”) that University [53]. Finally, the integrated experiment is setup by require expensive redesign and design iterations in industrial instantiating all components including the driver and the design processes, can be eliminated by encoding as constraints workflow specification according to the PCC testbench model. over static properties to navigate away from the erroneous In order to effectively manage the large demand of design configurations. For example, a “gradeability” simulations and analyses a job manager and dispatch server requirement (e.g. vehicle climbing a 20 degree hill can was created in the Execution Integration Platform. The accelerate at 2 m/s2) translates to an engine sizing problem and detailed description of the job manager is outside of the scope can be represented as the following constraint: of this paper. (Powerplant_maxTorque() * 1.3558) >= As the example illustrates, significant part of the overall ((Tire_radius()/1000) * (Vehicle_weight() + integration complexity, and complex transformation logic, is 13000) * sin(0.35) + ((Vehicle_weight() + 13000)/9.8) * 2) concentrated in the Model Composers. This underlines the significance of explicit modeling the semantics of Model Each of the four named properties referred to in the Composers using tools of the Semantic Backplane. constraint above are “variable” and are non-trivially dependent C. Static Design-Space Exploration Testbench upon the component selections. Satisfying this constraint in static DSE enforces that all “pruned-in” designs are Arguably, the ability to construct, shape and explore design conformant to the requirement, and will not result in “sizing spaces is critical to design of CPS, which has to fulfill a large mismatch” type of design errors. number of - often conflicting – performance objectives, as DESERT uses an AND-OR-LEAF tree structure to well as satisfy a variety of implicit and explicit constraints. represent design spaces in a generic manner, where AND The design space representation in CyPhyML, described nodes represent hierarchically inclusive containers, OR nodes earlier, using hierarchically layered assembly and component represent exclusive or choice containers, while LEAF nodes alternatives and parameterization is quite powerful in its represent design primitives. DESERT also allows properties ability to compactly represent extremely large design spaces. associated with LEAF nodes that can have unique value or a However, the size and dimensionality of the design space range of value assignment. DESERT supports a number of poses a significant challenge, rendering it infeasible to different types of constraints, specified using an extended exhaustively enumerate and evaluate all candidate designs Object Constraint Language (OCL) [62]. Functions supported encapsulated in the design space representation. in constraints include all logical, trigonometric and logarithm OpenMETA design flow, as we articulated earlier, employs functions as well as sign, rint, sqrt, and abs. DESERT’s a progressive deepening strategy i.e. evaluate the candidate internal operation and user interface is outside the scope of designs using lower fidelity models and progress toward this paper, however can be seen in [61]. increasingly complex, higher fidelity models and focus on The static design space exploration testbench maps rapidly decreasing the number of candidate designs. However, CyPhyML design spaces to DESERT’s AND-OR-LEAF even with this pragmatic strategy a major challenge was that representation with the following mapping – Component the initial combinatorial design space (sometimes with billions Assemblies are mapped to AND nodes, Alternative Containers of candidate designs) is simply infeasible even to enumerate are mapped to OR nodes, and Components are mapped to let alone evaluate irrespective of the model fidelity. Therefore, LEAF nodes. Constraints, as exemplified above, are we had to employ powerful analytic techniques that could represented in CyPhyML using multiple graphical and textual manipulate entire design spaces without exhaustively notations amenable to end-users. In the mapping process, enumerating them. Inspired by success in model checking, these constraints are translated into the extended OCL notation ([56]), our prior work on design space exploration in supported by DESERT. In CyPhyML inter- and intra- hardware-software co-design [57][58], resulted in component property relations are represented with “value development of DESERT [59][60][61] – a tool for constraint- flows”, while in translation to DESERT these “value flows” guided pruning of design spaces using symbolic methods. are mapped into constraints. The static design space exploration testbench utilizes and DESERT accepts as input the design space representation as integrates DESERT into the OpenMETA design flow. The an AND-OR-LEAF tree, a set of constraints, and generates an static design space exploration Testbench is uniquely distinct enumeration of satisfying configurations. These configurations (compared to other testbenches in OpenMETA), in that it represent a binding of choice elements (OR-nodes) to the transforms entire design space (rather than design specific selection (either an AND-node or a LEAF-node). In configurations) into a symbolic representation suitable for the inverse transformation these configurations are elaborated DESERT, and then inverse transforms the (severely) trimmed back into hierarchical component assemblies in CyPhyML’s design space from DESERT. native representation, where they become suitable for We refer to this as “static design space exploration”, since subsequent evaluation by other testbenches. DESERT solves constraints over static (time-invariant) Design space is a foundational construct in OpenMETA, properties of design elements (e.g. MaxTorque, or MaxRPM and our experience indicated the need to robustly support the of an Engine) and their analytic composition over design 0250-SIP-2017-PIEEE 17 construct throughout the design process with additional tools. with simulation that is achieved through model integration and For example, designers in OpenMETA after an iteration the CyPhyML modeling integration language. through the design flow often wanted to understand the critical Formal verification methods are integrated in OpenMETA component selections i.e. components that occur in most using verification testbenches. The verification testbenches viable design configurations. We also observed that the design include (1) system models in some mathematical domain (e.g., flow is not linear even in the sense that the design space is not differential algebraic equations (DAEs) [29], qualitative fully defined (depth wise or breadth wise) all at once in the automata [49], and hybrid systems) [48]; (2) requirements beginning, but rather follows an iterative path that can best be expressed as model properties in some logic framework [64]; articulated as alternately narrowing – by exploration and (3) information about ranges of inputs, model parameters, pruning; deepening – by further elaborating and adding details context, and initial conditions; and (4) algorithms proving that in subset of component hierarchies; and widening – by adding the models satisfy the properties over the defined domains or additional component selections and parameterization in sub- generating counterexamples demonstrating that the properties trees. These observations led us to improve support for design are violated. space management by creating several helper tools – Design The main modeling formalism used in the lumped Space Refiner, Design Space Refactorer, Design Space parameter dynamics abstraction level in OpenMETA is hybrid Criticality Meter [61]. Finally, DESERT as a symbolic systems [50]. A significant challenge is the automated constraint satisfaction tool represents a powerful capability translation from composed simulation models represented in however is constrained by the underlying BDD and MT-BDD the Modelica language to mathematical models that can be backend and limited in terms of algebraic domains supported. used for verification and formal analysis. In OpenMETA, this The recent development in SMT solvers, particularly with an problem of automated translation is decomposed to multiple expansive support library of domain theories, offers a unique subproblems that are addressed by different tools: opportunity to augment DESERT with an SMT solver based 1. The first step is the construction of a component model backend library consisting only of declarative models that can be represented with mathematical equations. We developed a D. Verification Testbenches fully equation-based, symbolic version of the FANG Verification is a key enabler to the development of Modelica component libraries that the formal verification advanced CPS by improving system assurance with tools are able to process. compressed qualification times [63]. Verification aims at 2. The second step is the use of tools within the Modelica analyzing functional and behavioral correctness in the compiler for generating the mathematical equations of presence of multiple sources of non-determinism and has a complex system models with interconnections of multiple crucial role in model-based design. The ultimate objective is components. These models are used as inputs to model to facilitate the “correct-by-construction” design of systems by translators that generate formal models suitable for reducing the need for expensive and unavoidably incomplete various verification techniques. Although the coverage of experimental prototyping and testing. Verification methods automated translation is not complete, the models and involve analysis and reasoning using models of the system in tools developed, demonstrate the feasibility of the each step of the design to ensure that it satisfies the approach for a large and important class of system requirements. Such methods are gaining increased attention dynamics. because they allow the decomposition of the design process The use of verification methods in the design flow starts into an iterative progression from requirement models to with the construction of a system model that is adapted to the implementation models using the repeated step of model property that needs to be verified. In the context of construction, verification, and transformation [2][3]. In this OpenMETA, examples of such properties express vehicle process, verification techniques can focus on the semantics of requirements that can be associated with mobility operational a suite of abstract system models that are incrementally range, payload, braking, and other performance requirements. refined into implementation models. In addition to performance, safety requirements comprise Verification of CPS is a very hard problem due to another group of properties to be verified. For example, a heterogeneous, highly nonlinear and nondeterministic typical requirement for military vehicles is that the vehicle dynamics, scalability due to large number of components, and shall meet all performance targets for all load conditions complexity of formal framework needed to express and reason without exceeding component manufacturers limits. about requirements. These challenges are addressed typically Such safety properties are associated with subsystems and by considering simplified abstractions and/or approximations components and need to be derived during the system design. of system models. A fundamental gap is the lack of Verification methods require encoding such properties in a understanding of the interrelations among different formal logic framework. In OpenMETA, linear temporal logic abstractions as well as the lack of methods for the automated (LTL) is used to express performance and safety properties. composition of multi-abstraction and multi-fidelity models LTL formulae use combinations of temporal and logic that can be adapted to the property to be verified. A significant operators to express complex properties that must be satisfied objective in our work is the development of methods and tools by the system behavior [50]. Writing the informal system for automated generation of suitable abstractions from detailed requirements as formal temporal logic formulas is a systems models that enable the use of formal verification. An challenging task especially for non-experts. The project additional objective is the integration of verification methods developed a template-based tool that allows users to write properties using an intuitive interface, which then are 0250-SIP-2017-PIEEE 18 translated to LTL formulas that are used by the verification abstraction based tool for verifying composed controller and tools. Reliability requirements comprise another class of plant models. Examples of verification testbenches include: system requirements that need to be addressed. When a system 1. A drivetrain model with transmission controller in order is deployed, faults occur in the field due to environmental to determine the range of grades of the road where gears conditions, poor maintenance, and attacks on equipment. provably do not chatter for all driver throttle positions Therefore, it is crucial to reason about fault behavior and and, failure impact as part of the early system correctness analysis 2. A model of an engine, which takes as input torque value and verification efforts, and determine the survivability and and outputs engine RPM, in order to determine the range continued utility of equipment in the presence of faults of inputs that guarantees that the engine RPM is bounded. [51][52]. The main limitations are related to the coverage of the The tool chain includes three verification methods for slicer and model translator of the physical plant. complementing simulation-based analysis: (1) qualitative The goal of the reliability analysis testbenches is to evaluate reasoning, (2) verification based on relational abstractions, and a vehicle design using various reliability metrics. The (3) reliability analysis using fault-augmented models. These testbenches use fault augmented component models that are methods provide different but synergistic capabilities, developed to simulate the effects of component faults and their demonstrate how verification methods can be integrated in the propagation to system-level behavior [51]. The starting point design tool-chain, and how they can be used for reducing the of the reliability analysis is a design with fault-augmented need for prototyping and testing. component models [52]. Components are associated with Qualitative reasoning (QR) aims to automate verification of applicable damage parameter maps that provide the continuous dynamics by generating qualitative descriptions of probability distributions of damage parameters (e.g., wear physical systems and their behaviors [49]. QR can identify fraction for friction wear in a clutch) with respect to usage as feasible designs, suggest parameter changes that improve well as other component-specific parameters (e.g., geometric design quality, and highlight potential problems due to poor constant and maximum normal force for a clutch). The design parameter choice relying on qualitative abstractions of the is then analyzed and the damage-parameter maps are system dynamics. In OpenMETA, Modelica hybrid system interpolated to generate time/usage dependent probabilistic models are abstracted into constraint networks that are used to transitions from nominal to faulty state for the system generate a multi-trajectory simulation, or envisionment, of all components. Subsequently, the design is run through a suite of possible qualitative behaviors of the abstraction. This reliability testbenches. These testbenches evaluate the design envisionment enables qualitative verification where against several key reliability requirements as they effect behavioral/safety requirements can be proved to be satisfied system-level performance after extended usage. for all, some, or no choice of the parameters in the underlying In order to complement formal verification, a suite of Modelica model. Qualitative analysis can be performed early methods for performing probabilistic analysis for a given in the design process before all parameters have been selected. performance requirement are also developed and integrated. In the case the model may not meet the requirements for a While deterministic methods seek to offer a yes/no answer to specific choice of design parameters, QR provides guidance verification questions, probabilistic methods provide a about how to modify the model parameters. We were able to probabilistic certificate of correctness (PCC) using methods of analyze models with more than a hundred equations and uncertainty quantification (UQ). The goal is to determine the integrate the analysis into the tool chain to evaluate various probability that the performance function is less than (or system requirements. The main limitation is scaling up to greater than) the requirement. PCC methods extend simulation more complex models consisting of 1000s of equations that methods by assuming that inputs, outputs, and model are typical of complete vehicle systems models. parameters are uncertain but can be specified using probability The tool suite uses the HybridSal Relational Abstraction as distributions. In general, they are more scalable than formal a formal verification tool for verifying safety properties of verification methods but they require additional information in CPS [50]. The main objective is verifying control algorithms order to quantify the uncertainty in the model. PCC while taking into account the behavior of the physical plant. testbenches use Modelica system models extended to capture The controller is modeled using Matlab, Simulink and uncertainty in the system inputs and parameters. As part of our Stateflow while the physical plant is modeled in Modelica project, we implemented six UQ methods for PCC estimation: using components from the declarative component library. Monte Carlo Simulation (MCS), Taylor Series Method (TSM), Physical plants are represented using differential algebraic Most Probable Point (MPP), Full Factorial Numerical equations (DAEs). The desired behavior of the controller is Integration (FFNI), Univariate Dimension Reduction (UDR), encoded in LTL formulas focusing on safety properties. and Polynomial Chaos Expansion (PCE) [53]. In addition, we Integration is based on (1) a slicer which aims at identifying a implemented sensitivity analysis methods to quantify the small subset of the set of DAEs of the Modelica model that amount of variance that each input factor contributes to the correctly describe the dynamics of a given set of variables PCC of the performance requirement (Sobol, FAST, EFAST which are selected as the input variables of the controller [54]). models, (2) a model translator from Modelica to the HybridSal In conclusion, we developed and integrated in the verification tool, (3) a tool that takes as input a Matlab OpenMETA toolchain two formal verification methods, one controller model and outputs the same controller in reliability analysis method, and methods for uncertainty HybridSal’s input language, and (4) the HybridSal relational quantification. The developed tools have been demonstrated using various testbenches representing realistic vehicle 0250-SIP-2017-PIEEE 19 models. Although the model complexity, heterogeneity, and Modelica Standard Library (MSL)8 is a crowdsourced, multi- scale create considerable challenges, verification methods physics lumped parameter dynamics library developed and provide complementary capabilities for addressing maintained by the OpenModelica Consortium; Modelon’s performance, safety, and reliability requirements at different Vehicle Dynamics Library9 is a COTS component library on phases of the design flow. These capabilities facilitate the top of the Modelica Standard Library; and many others. “correct-by-construction” design and can be used to We believe that domain specific model libraries will continue complement simulation methods. emerging both in open-source and COTS form and will become one of the engines in the progress of component and V. LESSONS LEARNED model-based design. We believe that our CCM component Creating end-to-end design automation tool chains for the models should accelerate progress in creation of multi-domain model- and component-based design of CPS is a significant component model libraries. challenge that extends to the fundamentals of compositionality C. Automated Model Composition in heterogeneous domains, formulating abstractions with This is frequently missing in physical domains due to the established relationships, modularization of models to create perception that useful physical models need to be hand-crafted reusable component model libraries, verification methods and for specific phenomena, and consequently, the modeling scaling to real-life systems. The OpenMETA project, that was abstractions used are often decoupled from the physical part of DARPA’s AVM program offered unique opportunity architecture of the system. One explanation for this is the for conducting a large-scale integration experiment with frequent use of modeling approaches that do not support extensive testing of the results using a design competition [8]. generic compositionality. Our CCM component model places Below we summarize some of the most important lessons strong emphasis on compositional semantics to resolve this learned. problem. However, we have identified several challenges, A. Horizontal Integration Platforms ranging from technical to fundamental that had to be Approaching the design automation challenge by addressed or still open according to our knowledge. establishing horizontal model, tool and execution integration Target language challenge. While CyPhyML incorporates platforms was a necessity. The key insight for us was that the an extensive set of checks on the well-formedness of the tradeoffs among using standard-based, or tool/vendor-specific, composed architecture models (as defined in the structural or locally defined ontologies and DSMLs as model integration semantics of CyPhyML), when the Model Composers languages is an interesting problem of its own and the answer generate the individual analysis models there are additional depends on the context: characteristics of the CPS product considerations that are specific to the target language and do line, size of the company and complexity of the engineering not belong to the CyPhyML structural semantics. For process. However, independently from the context, we did not example, the composed Modelica models need to satisfy a conclude that the ultimate solution would be the emergence of range of best practices and conventions that need to be a standard, universal model integration language (e.g. guaranteed by component authoring guidelines and curation SysML4, or other), simply, because these languages need to checks. These guidelines include rules such as using potential respond rapidly to changes in the product line and in the rather than flow as boundary condition in physical models, engineering processes, and the cost of evolving domain- preference of using declarative constructs instead of specific modeling languages is not a prohibiting factor due to imperative, and many others. Even with these guidelines, the appearance of metaprogrammable modeling and model extensive testing and debugging of component models in management tools such as WebGME5, EMF6 and others. We various system contexts were required to achieve acceptable found that the use of model and tool integration platforms can composability. provide an increased level of decoupling between the product Component validity challenge. Models of physical and engineering process specific view of the systems’ components have (ocassionally implicitly) an operating companies and the more generic view of tool vendors. regime where they are valid i.e. reproduce accurately the behavior of the referent (component or subsystem). In lumped B. Availability of Reusable Component Model Libraries parameter dynamic models this region of validity is usually In model- and component-based design the key expressed as constraints over the state variables. The challenge productivity factors depend on the availability of reusable, to composability is that, whether or not component models componentized model libraries on different domains and on remain valid during simulation runs depends on their the feasibility of fully automated model composition during interaction with other components. It is possible that a design space exploration. Both of these factors were much computed trajectory becomes invalid simply because one or harder to obtain than we expected. There are excellent more components temporarily leave their region of validity, or examples for existing, highly successful model libraries, both in other words, they lose their composability. We addressed in crowdsourced or COTS form: DOE’s EnergyPlus7 is an this problem by making the regions of validity explicit i.e. part open-source model and simulation library for energy; the of the component models, and by actively monitoring validity violations these during simulation runs.

4 http://www.omgsysml.org/ 5 http://webgme.org 8 http://www.modelica.org 6 http://www.eclipse.org/modeling/emf/docs/ 9http://www.modelon.com/products/modelica-libraries/vehicle-dynamics- 7http://apps1.eere.energy.gov/buildings/energyplus/energyplus_addons.cfm library/ 0250-SIP-2017-PIEEE 20

Complexity challenge. A particularly hard problem of integration of effective and usable verification methods into automated composition of system models from multi- the design flow. The fundamental problems for addressing phenomenon component models is that it can easily produce such challenges can be categorized along the following four very complex, high-order models that the solvers cannot dimensions: (1) complexity due to highly nonlinear and handle. We experimented with supporting multiple nondeterministic dynamics, (2) heterogeneity of components abstractions, and multiple fidelity levels in component and behaviors, (3) scalability due to large number of libraries, and with adapting the selected level of abstraction components, and (4) epistemic uncertainty due to neglecting and component fidelity level to the system property being certain effects in the models because of knowledge gaps. analyzed. While this approach has proved to be promising, Furthermore, these challenges are coupled in modern systems the cost of creating multiple fidelity models has remained an and this coupling magnifies the problem. Another fundamental open challenge. Deriving surrogate models combined with gap is the lack of understanding of the interrelations among machine learning [67][68] was a possible solution but different abstractions and the lack of methods for the validation of surrogate models was still costly, and automated composition of multi-abstraction and multi-fidelity composability of surrogate models is not well understood. models that are adapted to the property to be verified. Time resolution challenge. Complex, multi physics models F. Code complexity frequently incorporate dynamics with highly different frequency ranges. For example in drivetrains, the time The dominant challenge in developing OpenMETA was constant of mechanical processes is significantly smaller than integration: models, tools and executions. The OpenMETA that of thermal processes. Composition of the system level integration platforms included over 1M lines of code that is Modelica model for a drivetrain yields a large number of reusable in many CPS design contexts. In the AVM project, equations for which the simulation with a single Modelica OpenMETA integrated 29 open source and 8 commercial simulator may be extremely slow because the step size of the tools, representing a code base estimated to be 2 orders of solver is determined by the fastest processes. We magnitude larger than OpenMETA [6] itself. The conclusion experimented with phenomenon-based slicing of multi physics is that integration does matter. It is scientifically challenging models by first composing the system-level model using and yields major benefits. This is particularly true in design component models, followed by the decomposition of the automation for CPS, where integrated design flows are still model - but not along the component/subsystem boundaries, not a reality. rather along physical phenomena (mechanical processes and G. Evaluation and Transitioning thermal process) so that we can separate the fast and slow The OpenMETA tool suite evolved along parallel dynamics. This decomposition led to two models that can be evaluation efforts performed first in the FANG 1 design co-simulated using the co-simulation testbench, so the re- competition focusing on the drivetrain and mobility aspects of composition of the system-level simulation occurs in a different semantic domain (distributed co-simulation). We the vehicle and later on the hull design. A comprehensive have demonstrated the feasibility and impact of the approach summary of the FANG 1 competition and related evaluation is available in [8]. While FANG 1 was relatively early in the in [66]. However, the construction of model libraries that can project (after 2 years of effort), we believe that it has proved be automatically sliced by physical phenomena is an open early the feasibility and validity of the overall integration problem. architecture we discussed in this paper. The competition D. Semantic Backplane experience also provided our team useful insights and While our experience with constructing a formal semantic feedback regarding the maturity of the dominantly open model for key integration entities (model integration language, source simulation and verification component technologies, ontologies, semantic interfaces, model composers) has proved and in a more general sense, advantages and limitations of the to be extremely valuable to keep the overall integration effort model- and component-based design approach for CPS (the consistent, the fact that the “production tools” in OpenMETA most essential ones described above). (the metaprogrammable Generic Modeling Environment The OpenMETA transitioning process has started in 2014 10 (GME), its metamodels, and the code of the model composers) through various transitioning programs , partially by spinoff were separate from the FORMULA-2 based formal companies pursuing a wide range of domains including the specifications created the risk of divergence between the integration of tool chain pilots for electronic design, robotics, implemented integration components and their formal models. aerospace systems and automotive systems. Since OpenMETA To mitigate this risk, a tight coupling need to exist between addressed integration technologies that were motivated by the the metaprogrammable tools and FORMULA-2, which is an needs of CPS design, their applicability remains broad. Most active research effort. of the transitioning efforts utilize two open-source repositories in Github: one for the OpenMETA-core11 and one for E. Verification OpenMETA-extensions12. In spite of significant progress, verification of complex CPS remains a very hard problem and there are major challenges that hinder the use of verification methods: (1) automated translation of complex component-based models to suitable 10https://www.pddnet.com/news/2014/02/darpa-begins-early-transition- abstractions, (2) mapping of significant correctness adaptive-vehicle-make-technologies requirements to typical safety/reachability properties, and (3) 11 https://github.com/metamorph-inc/meta-core 12 https://github.com/metamorph-inc/openmeta-mms 0250-SIP-2017-PIEEE 21

VI. CONCLUSION AND FUTURE DIRECTIONS the design process that will require fundamental re- The OpenMETA integration platforms addressed the thinking its key elements from modeling to verification following problems of component- and model-based design and to assurance argumentation. . for CPS: (a) composing system-level models from reusable, heterogeneous component model libraries, (b) extending the limits of correct-by-construction design by supporting ACKNOWLEDGMENT heterogeneity and automation, (c) applying multiple level of We are grateful for the vision and leadership of Mr. Paul abstractions in design flows for CPS, (d) executing rapid Eremenko who conceived and led the program through the design trade-offs, (e) defining interface between design and crucial first two years. We also would like to express our manufacturing for CPS, and (f) creating an open framework appreciation to Dr. Nathan Wiedenman for steering the for reusing open-source tool assets. The project gave the program through the FANG-1 design challenge and Dr. Kevin OpenMETA developers unique opportunity not only to Massey for guiding the program to its conclusion. The understand the limits of the current state-of-the-art in the OpenMETA project integrated technology components context of a real-life DoD challenge problem but also to push provided by researchers from Vanderbilt, Georgia Tech, MIT, the limits in several areas. We believe that the program also Oregon State, SRI, PARC, and Modelon. Authors are also provided opportunity for the developers and the research grateful for the significant contributions of Mr. Adam Nagel, community in general to better understand open problems and Metamorph Inc. to the transitioning efforts and latest their impact on the broad applicability of model-based design extensions of OpenMETA. Our project benefited technologies. Based on this experience we summarize below tremendously from the discussions, feedback and ideas we some of the open challenges and opportunities. received from our team of collaborators, and from Dr. Mike Lowry, Chief Engineer, and Dr. Kirstie Bellman, Chief 1. Product and manufacturing process co-design. Scientist of the AVM program. We would also like to Merging the relatively isolated activities in product and recognize the importance of the guidance from the Senior manufacturing process design into an integrated co-design Strategy Panel including Alberto Sangiovanni Vincentelli (UC process promises the largest benefits and truly Berkeley), Prof. Joseph Sifakis (VERIMAG), Daniel Frey revolutionary advantages. This would be particularly (MIT), Karl Hedrick (UC Berkeley), Olivier De Weck (MIT), important with the increased use of composites in and Janos Sztipanovits (Vanderbilt). manufacturing, in which the interdependence of product models and manufacturing process models is large and not understood well. While the OpenMETA design space REFERENCES exploration process incorporated feedback from [1] S. A. Seshia, "Combining Induction, Deduction, and Structure for manufacturability analysis, full integration was not Verification and Synthesis," in Proceedings of the IEEE, vol. 103, achieved yet no. 11, pp. 2036-2051, Nov. 2015 2. Configurable domain-specific design environments. [2] Sanjit A. Seshia, Shiyan Hu, Wenchao Li, Qi Zhu, "Design Automation of Cyber-Physical Systems: Challenges Advances The OpenMETA horizontal integration platforms and Opportunities", Computer-Aided Design of Integrated emerged as key enablers for the tool chain development Circuits and Systems IEEE Transactions on, vol. 36, pp. 1421- effort. The primary end users of OpenMETA were the 1434, 2017, ISSN 0278-0070. vehicle designers. Consequently, the implemented [3] P. Nuzzo, A.L. Sangiovanni-Vincentelli, D. Bresolin, L. Geretti, and T. Villa, “A Platform-Based Design Methodology With automations and user interfaces served the designer Contracts and Related Tools for the Design of Cyber-Physical community. However, the development of the model, tool Systems,” in Proceedings of the IEEE, vol. 103, no. 11, pp. 2104- and execution integration platforms – the core 2132, Nov. 2015. contributions of OpenMETA – created opportunity for [4] Joseph Sifakis, “System Design Automation: Challenges and Limitations,” in Proceedings of the IEEE, vol. 103, no. 11, pp. automation and improved user interfaces for another 2093-2103, Nov. 2015. category of users, those, whose goal is to integrate [5] Eremenko, Paul: “Philosophical Underpinnings of Adaptive domain specific integrated CPS design tool chains. To Vehicle Make,” DARPA-BAA-12-15. Appendix 1, December 5, achieve progress in this area is one of our goals. 2011. [6] Sztipanovits, J., Bapty, T., Neema, S., Koutsoukos, X., Jackson, 3. Integrating model-based and data-driven design. E. “Design Tool Chain for Cyber Physical Systems: Lessons A fundamental limitation of model-based design Learned,” In Proceedings of DAC’15, pp. 106, June 07 - 11, processes is that models of physical components and 2015, San Francisco, CA, USA. environments always have epistemic uncertainties. [7] Ackerman, Spencer: "This Is the Million-Dollar Design for Darpa's Crowdsourced Swimming Tank". Wired. (22 April 2013) Epistemic uncertainties originate in the lack of knowledge (Retrieved 24 April 2013, https://www.wired.com/2013/04/darpa- or data. In complex CPS their weight and potential risk is fang-winner). significant because they decrease predictive properties of [8] Olivier L de Weck, Eun Suk Suh. “Complex System Design the design models and limit the confidence level of the through Crowdsourcing : DARPA FANG - 1 as a Case Study,” Industrial Engineering Magazine 21(4), 2014.12, 32-37, url: assurance arguments. Decreasing this uncertainty is http://www.dbpia.co.kr/Article/NODE06085424. expensive and with the rapidly growing size and openness [9] M. Broy, M. Feilkas, M. Herrmannsdoerfer, S. Merenda, and D. of important categories of CPS is becoming unfeasible. A Ratiu. Seamless model-based development: From isolated tools promising approach of addressing this challenge is the to integrated model engineering environments. Proc. of the IEEE, 98(4):526 –545, 2010. incorporation of data-driven machine learning methods in 0250-SIP-2017-PIEEE 22

[10] D’Ambrosio, J., Bapty, T., Jackson, E.,Paunicka, J., Sztipanovits, [27] http://research.microsoft.com/formula Comprehensive Transitioning Model for AVM Tools – Boeing, [28] Michael R. Myers, Richard E. Gregory, Joseph E. Hite, Sandeep GM Research, Microsoft Research, Metamorph, Vanderbilt Neema and Ted Bapty, “Collaborative Design in DARPA’s White Paper response to DARPA –SN-14-04, October 31, 2013. FANG I Challenge,” ASME 2013 International Mechanical [11] Pinto A, Bonivento A, Sangiovanni-Vincentelli AL, Passerone R, Engineering Congress and Exposition Volume 12: Systems and Sgroi M. System level design paradigms: Platform-based design Design San Diego, California, USA, November 15–21, 2013. and communication synthesis. ACM Transactions on Design [29] Adrian Pop, Martin Sjölund, Adeel Ashgar, Peter Fritzson, and Automation of Electronic Systems, 2004 Jun;11(3):537–563. Francesco Casella. Integrated Debugging of Modelica Models. [12] E. A. Lee, “CPS foundations,” in Proc. 47th Design Automation Modeling, Identification and Control, 35(2):93--107, 2014. Conference (DAC), 2010, Anaheim, CA, USA, Jul. 2010, pp. [30] https://openmodelica.org 737–742 [31] Jackson, E., Simko, G., Sztipanovits, J.: “Diversely Enumerating [13] Lattmann, Zs., Nagel, A., Scott, J., Smyth, K., vanBuskirk, C., System Level Architectures,” Proceedings of EMSOFT 2013, Porter, J., Neema, S., Bapty, T., Sztipanovits, J.: “Towards Embedded Systems Week, September 29-October 4, 2013 Automated Evaluation of Vehicle Dynamics in System-Level Montreal, CA. Design,” Proceedings of the ASME 2012 International Design [32] Eker, J.; Janneck, J.W.; Lee, E.A.; Jie Liu; Xiaojun Liu; Ludvig, Engineering Technical Conferences & Computers and J.; Neuendorffer, S.; Sachs, S.; Yuhong Xiong: “Taming Information in Engineering Conference IDETC/CIE 2012 August heterogeneity – the Ptolemy approach,” Proceedings of the IEEE, 12-15, 2012, Chicago, IL Vol. 91, No.1., pp. 127-144, January, 2003. [14] Simko, G., Levendovszky, T., Neema, S., Jackson, E., Bapty, T., [33] Lee, E.A., Seshia, S.A. “Introduction to Embedded Systems,: A Porter, J., Sztipanovits, J.: “Foundation for Model Integration: Cyber-Physical System Approach,” 2nd. Ed. LeeSeshia.org. 2015. Semantic Backplane” Proceedings of the ASME 2012 [Online] Available: http://leeseshia.org International Design Engineering Technical Conferences & [34] Simulink: http://www.mathworks.com/products/simulink/ Computers and Information in Engineering Conference [35] J. Porter, G. Hemingway, Nine, H., van Buskirk, C., Kottenstette, IDETC/CIE 2012 August 12-15, 2012, Chicago, IL N., Karsai, G.,and Sztipanovits, J. The ESMoL language and [15] Wrenn, R., Nagel, A., Owens, R., Yao, D., Neema, H., Shi, F., tools for high-confidence distributed control systems design. part Smyth, K., vanBuskirk, C., Porter, J., Bapty, T., Neema, S., 1: Design language, modeling framework, and analysis. Tech. Sztipanovits, J., Ceisel, J., Mavris, D.: “Towards Automated Report ISIS-10-109, ISIS, Vanderbilt Univ., Nashville, TN, 2010 Exploration and Assembly of Vehicle Design Models,” [36] Functional Mock-up Interface – www.fmi-standard.org Proceedings of the ASME 2012 International Design Engineering [37] Modelica Association: Modelica – A Unified Object-Oriented Technical Conferences & Computers and Information in Language for Physical Systems Modeling. Language Engineering Conference IDETC/CIE 2012 August 12-15, 2012, Specification, Version 3.2, March 24, 2010: Chicago, IL www.modelica.org/documentas/ModelicaSpec32.pdf [16] G. Karsai, A. Ledeczi, S. Neema, and J. Sztipanovits, “The [38] Karnopp, Dean C., Margolis, Donald L., Rosenberg, Ronald C., Model-Integrated Computing Toolsuite: Metaprogrammable 1990: System dynamics: a unified approach, Wiley, ISBN 0-471- Tools for Embedded Control System Design,” 2006, pp. 50–55 62171-4. [17] Alberto Sangiovanni-Vincentelli, Sandeep Shukla, Janos [39] Jan C. Willems: “The Behavioral Approach to Open and Sztipanovits, Guang Yang: “Metamodeling: An Emerging Interconnected Systems,” IEEE Control Systems Magazine, pp. representation Paradigm for System-Level Design,” IEEE Design 46-99, December 2007. and Test of Computers May/June 2009 [40] Siyuan Dai and Xenofon Koutsoukos. "Model-Based Automotive [18] http://openmdao.org Control Design Using Port-Hamiltonian Systems", International [19] IEEE 1516.1–2010 – Standard for Modeling and Simulation High Conference on Complex Systems Engineering (ICCSE 2015), Level Architecture – Federate Interface Specification University of Connecticut, November 9-10, 2015. [20] IEEE 1516–2010 – Standard for Modeling and Simulation High [41] A. Sangiovanni-Vincentelli, W. Damm, and R. Passerone, Level Architecture – Framework and Rules,” pp. 1–38, 2010. “Taming Dr. Frankenstein: Contract-based design for cyber- [21] Juracz, L., Z. Lattmann, T. Levendovszky, G. Hemingway, W. physical systems,” European. Journal of Control, vol. 18, no. 3, Gaggioli, T. Netterville, G. Pap, K. Smyth, and L. Howard. pp. 217–238, 2012. [Online]. “VehicleFORGE: A Cloud-Based Infrastructure for Collaborative http://www.sciencedirect.com/science/article/pii/S094735801270 Model-Based Design,” Proceedings of the 2nd International 9433 Workshop on Model-Driven Engineering for High Performance [42] Gabor Simko, Janos Sztipanovits: “Model Integration in CPS,” in and Cloud Computing (MDHPCL 2013). Co-located with 16th Raj Rajkumar, Dioniso de Niz, Mark Klein (Ed.) Cyber Physical International Conference on Model Driven Engineering Systems, pp 331-360, Addison-Wesley (2017) ISBN-13:978-0- Languages and Systems (MODELS 2013) l. Vol-1118. Chapter 321-92696-8 252014 [43] Simko, G., Lindecker, D., Levendovszky, T., Neema, S., (url:http://www.isis.vanderbilt.edu/sites/default/files/MDHPCL% Sztipanovits, J.: “Specification of cyber-physical components 202013%2004-paper.pdf with formal semantics - integration and composition. In: [22] Sangiovanni-Vincentell, Alberto: Quo Vadis, SLD? Reasoning ACM/IEEE 16th International Conference on Model Driven About the Trends and Challenges of System Level Design, Engineering Languages and Systems (MODELS 2013) Proceedings of the IEEE, vol. 95, No. 3, pp 467-506, March 2007 [44] Simko, G., Lindecker, D., Levendovszky, T., Jackson, E.K., [23] Nuzzo, P.; Xu, H.; Ozay, N.; Finn, J. B.; Sangiovanni- Neema, S., Sztipanovits, J.: A framework for unambiguous and Vincentelli, A. L.; Murray, R. M.; Donzé, A.; Seshia, S. A, “A extensible specification of DSMLs for cyber-physical systems. contract-based methodology for aircraft electric power system In: IEEE 20th International Conference and Workshops on the design.,” in EEE Access, Vol. 2, p. 1-25. January 01, 2014. Engineering of Computer Based Systems (ECBS 2013) [24] Jackson, E., Sztipanovits, J.: ‘Formalizing the Structural [45] Bjørner, N., Ken McMillan and Rybalchenko. A., Higher-order Semantics of Domain-Specific Modeling Languages,” Journal of Program Verification as Satisfiability Modulo Theories with Software and Systems Modeling Volume 8, Issue 4, pp. 451-478, Algebraic Data-types. In informal proceedings of HOPA 2013 September 2009. (Workshop on Higher-Order Program Analysis). [25] Kai Chen, Janos Sztipanovits, Sandeep Neema: “Compositional [46] Desai, A., Vivek G., Jackson, E., Qadeer, E. Rajamani, S., and Specification of Behavioral Semantics,” in Design, Automation, Zufferey, D.:. P: Safe asynchronous event-driven programming. and Test in Europe: The Most Influential Papers of 10 Years In Proceedings of ACM SIGPLAN Conference on Programming DATE, Rudy Lauwereins and Jan Madsen (Eds), Springer 2008. Language Design and Implementation (PLDI 2013). [26] E. K. Jackson, T. Levendovszky, and D. Balasubramanian, [47] Jackson, E.: Engineering of domain-speciifc languages with “Reasoning about Metamodeling with Formal Specifications and FORMULA 2. HILT '13 Proceedings of the 2013 ACM SIGAda Automatic Proofs,” in Model Driven Engineering Languages and annual conference on High Integrity Language Technology. Systems, vol. 6981, J. Whittle, T. Clark, and T. Kühne, Eds. pp. [48] P. Antsaklis (Ed.). Proceedings of the IEEE, Special issue on 653–667, Berlin, Heidelberg: Springer Berlin Heidelberg, 2011. hybrid systems: Theory and Applications, 88(7), 2000. 0250-SIP-2017-PIEEE 23

[49] D.S. Weld and J. De Kleer (Eds.). Readings in qualitative Batteh, Hubertus Tummescheit, Chandrasekar Sureshkumar: reasoning about physical systems. Morgan Kaufmann, 2013. “Model-Based Integration Platform for FMI Co-Simulation and [50] Ashish Tiwari. "HybridSAL relational abstracter." Computer Heterogeneous Simulations of Cyber-Physical Systems,” Aided Verification. Springer Berlin/Heidelberg, 2012. Proceedings of the 10th International Modelica Conference, pp. [51] de Kleer, Johan, et al. "Fault augmented Modelica models." The 235-245, March 10-12, 2014, Lund, Sweden 24th International Workshop on Principles of Diagnosis. 2013. [67] Gorissen, D., Couckuyt, I., Demeester, P., Dhacne, T. “A [52] Honda, Tomonori, et al. "A simulation and modeling based Surrogate Modeling and Adaptive Sampling Toolbox for reliability requirements assessment methodology." ASME 2014 Computer Based Design,” Journal of Machine Learning Research International Design Engineering Technical Conferences and 11 (2010) 2051-2055 Computers and Information in Engineering Conference. [68] Ounpraseuth, S. “Gaussian Processes for Machine Learning,” American Society of Mechanical Engineers, 2014. Journal of the American Statistical Association, 103:481, 429- [53] Hoyle, C., et al. “Multi-Stage Uncertainty Quantification for 429, 2008. DOI: 10.1198/jasa.2008.s219 Verifying the Correctness of Complex System Designs”. ASME [69] Yakov Ben-Haim, “Order and Indeterminism: An Info-Gap 2011 International Design Engineering Technical Conferences. Perspective,” in M. Boumans, G. Hon and A. Petersen (eds.) 2011. Washington DC: ASME. Error and Uncertainty in Scientific Practice, pp. 157-175 [54] A. Saltelli, S. Tarantola, and F. Campolongo. "Sensitivity Pickering and Chatto Publishers, London, 2014. analysis as an ingredient of modeling." Statistical Science, 377- [70] http://openmdao.org 395, 2000. [55] Lattmann, Z. “An Analysis-Driven Rapid Design Process for Cyber-Physical Systems”. PhD thesis. Vanderbilt University. 2016. [56] Bryant R., “Symbolic Manipulation with Ordered Binary Decision Diagrams,” School of Computer Science, Carnegie Mellon University, Technical Report CMU-CS-92-160, July 1992. [57] Bapty, T., Neema S., Scott J., Sztipanovits J., and Asaad S., "Model-Integrated Tools for the Design of Dynamically Reconfigurable Systems," VLSI Design, vol. 10, no. 3, pp 281— 306, 2000. [58] Mohanty, S., Prasanna V., Neema S., and Davis J., “Rapid Design Space Exploration of Heterogeneous Embedded Systems using Symbolic Search and Multi-Granular Simulation,” Workshop on Languages, Compilers, and Tools for Embedded Systems (LCTES), ACM SIGPLAN Notices, vol. 37, issue 7, pp 18-27, Berlin, Germany, 2002. [59] Neema, S., Sztipanovits, J., Karsai, G, Butts, K.: „Constraint- Based Design Space Exploration and Model Synthesis,“ Proc. of EMSOFT’2003, Philadelphia, PA, October 2003. [60] Neema, H., Z. Lattmann, P. Meijer, J. Klingler, S. Neema, T. Bapty, J. Sztipanovits, and G. Karsai, "Design Space Exploration and Manipulation for Cyber Physical Systems", IFIP First International Workshop on Design Space Exploration of Cyber- Physical Systems (IDEAL' 2014), Berlin, Germany, Springer, 04/2014. [61] Lattmann Z., Pop A., Kleer J., Fritzson P., Janssen B., Neema S., Bapty T., Koutsoukos X., Klenk M., Bobrow D., Saha B., Kurtoglu T. (2014). “Verification and Design Exploration through Meta Tool Integration with OpenModelica”. In: Proceedings of the 10th International Modelica Conference. Lund University, olvegatan 20A, SE-223 62 LUND, SWEDEN: Modelica Association and Linoping University Electronic Press, pp. 353–362 [62] Jordi CabotMartin Gogolla: Object Constraint Language (OCL): A Definitive Guide. In Marco Bernardo, Vittorio Cortellessa, Alfonso Pierantonio (Eds.) Formal Methods for Model-Driven Engineering, 12th International School on Formal Methods for the Design of Computer, Communication, and Software Systems, SFM 2012, Bertinoro, Italy, June 18-23, 2012. Springer LNCS Vol. 7320.. [63] Lee, Edward A. "Cyber physical systems: Design challenges." In Object oriented real-time distributed computing (ISORC), 2008 11th ieee international symposium on, pp. 363-369. IEEE, 2008. [64] Emerson, E. Allen. "Temporal and modal logic." In Handbook of Theoretical Computer Science, Volume B: Formal Models and Sematics (B) pp. 99-1072, MIT Press Cambridge, MA, USA, 1990. [65] Sztipanovits, J., Bapty, T., Lattmann, Zs., Neema, S.: Composition and Compositionality ion CPS. In Griffor, E., (Ed) Handbook of System Safety and Security: Cyber Risk Management, Cyber Security, Threat Analysis, Functional Safety, Software Systems, and Cyber-Physical Systems. Elsevier, 2017 [66] Himanshu Neema, Jesse Gohl, Zsolt Lattmann, Janos Sztipanovits, Gabor Karsai, Sandeep Neema, Ted Bapty, John