The Representations and Practices of the Discipline of Systems Engineering

2016 Conference on Systems Engineering Research The representations and practices of the discipline of systems engineering

Stephen B. Johnson*

Department of Mechanical and Aerospace Engineering, University of Colorado, Colorado Spings, Colorado Springs, CO 80918, USA Dependable System Technologies, LLC, Broomfield, CO 80020

Abstract

This paper is the second of two publications that define the motivations, theoretical foundations, representations, and processes for a formalization of systems engineering (SE) called the “Discipline of Systems Engineering (DSE).” The first paper, called “Theoretical Foundations for the Discipline of Systems Engineering,” describes the problems of Traditional Systems Engineering (TSE) that this new approach strives to improve upon, the core ideas of the new approach summarized as a set of theoretical “bases,” and then derives a set of principles from these bases. From these bases and principles, this paper defines the goals and strategies of the new approach. It then describes the representations required for the new approach. Finally, this paper explains how to use these representations to reproduce with higher quality some typical products of current SE, and to create new knowledge and products that TSE either has difficulty producing, or does not produce.

Keywords: state; model; goal; requirement; control; value; knowledge; constraint; preference; intention; organization; institution; failure; anomaly; system health management; traceability.

1. Introduction This paper is the second of two publications that define the motivations, theoretical foundations, representations, and processes for a formalization of systems engineering (SE) called the “Discipline of Systems Engineering (DSE).” The first paper, called “Theoretical Foundations for the Discipline of Systems Engineering,”1 describes the problems of Traditional Systems Engineering (TSE) that this new approach strives to improve upon, the core ideas of the new approach summarized as a set of theoretical “bases,” and then derives a set of principles from these

* Tel.: 720-887-6246 E-mail address: [email protected]

® 2016 Stephen B. Johnson Stephen B. Johnson 2 bases. From these bases and principles, this paper begins by defining the goals and strategies of the new approach. It then describes the representations required for the new approach. Lastly, this paper explains how to use these representations to reproduce with higher quality, lower cost, and reduced schedules some typical products of current SE, and to create new knowledge and products that TSE either has difficulty producing, or does not produce. The “Theoretical Foundations” paper is assumed as a foundation for the concepts, models, and processes described in this publication. That paper argues that with ever-increasing system complexity, TSE provides insufficient system dependability, which is directly related to development cost overruns and schedule slips. This is a symptom of its dependence on natural language documentation. By contrast, classical engineering disciplines (CEDs) such as control theory and electrical engineering use formal physical/mathematical theories applied to the system in question through in models and simulations. Within their domains, CEDs produce results that are typically much more reliable, on schedule and within budgets much more frequently than occurs for the entire system when integrated by TSE. Postulating that this difference is largely due to the formal nature of CEDs and how this improves communication compared to TSE, DSE strives to emulate CEDs. To do this, DSE uses seven theoretical bases: Systems Theory, Value Theory, Model, State, Goal, Control, and Knowledge. From these DSE elaborates a suite of 25 principles, which will not be listed here. However, these will be used to form a set of strategies, representations, and practices in this paper.

2. Predictive Basis and Principles Since the publication of the Theoretical Foundations paper, one new “basis” has been identified: the “predictive basis.” The purpose of engineering, to paraphrase Karl Marx, is not to understand the world, but to change it. Engineering is a means by which a vision of a novel product that does not yet exist is created and built. As engineering inherently intends to create a “new future,” predictions about that future are inherent and essential. Several kinds of prediction are common, particularly for the familiar quad of cost, schedule, reliability (or more generally, dependability), and performance. As the future is inherently uncertain, a direct implication is the use of probabilistic techniques that can account for uncertainties about the system, its environment, and its “internal and external contexts.” These “contexts” refer to the organizations that design and build the product (the “internal” context) and external factors as policy, law, economics and culture that influence how the product is built and used. Conversely, the new product can influence these external factors, but cannot control them. As this “predictive basis” is new to the theory of DSE (though obviously not new in TSE or engineering in general), the principles are also new. Two have been postulated. These include the following.  Prediction of the future is necessary to enable control of the means by which the envisioned future system is realized.  Typical predictions required for DSE are of performance, dependability, cost, and schedule.

3. The purposes of systems engineering Historically, engineers created SE primarily to reduce the probability of complex system failure, and secondarily to create a better or ideally optimal design to achieve a system’s goals, all while attempting to develop the system as quickly as possible.2,3 Later, SE was integrated with systems management, which also supported management’s desire for predictable costs and schedules. TSE and DSE have similar purposes, which are denoted next.  A primary purpose of SE is to optimize system performance against given system preferences and constraints.  A primary purpose of SE is to integrate knowledge across institutional and disciplinary boundaries.  A primary purpose of SE is to prevent system failure.  A primary purpose of SE is to develop a system as quickly as possible, given a set of resources.  A primary purpose of SE is to support effective cost and schedule prediction.

4. The strategies of DSE With the purposes, bases, principles, and goals described above, DSE applies a variety of strategies, which are described in this section. Stephen B. Johnson 3

4.1. System theory strategies DSE applies the following strategies derived from system theory.

 DSE divides its space of representation into the system, the system’s environment, and the system’s internal and external contexts.  In hierarchical representations, DSE concepts are typically applied recursively to each level of the hierarchy.

The system is the item being designed, assessed, built, and operated. It is the entity engineered to achieve one or more purposes. The environment is the physical, logical, and human environment in which the system is operated. The internal context includes the organizations that design, assess, build, verify and validate the system; these are controllable by project management and systems engineers. The external context refers to organizations that provide guidance and resources to these organizations, and other factors often beyond direct control of any organization, such as economic and political influences and constraints. Over the life of a system, there can be changes to the system itself, to its operational environment, and to its internal and external contexts. All of them influence a system’s purposes, and to the judgment of how well or poorly those purposes are being achieved. The recursive strategy is typical of systems. One frequently finds the same idea, such as what “the system” actually is or what constitutes cause or effect, being applied in different ways to the same physical components or behaviors. This is often due to people having control of, or being interested in different parts of the system. As an example, for an organization that builds a system component, that component is “the system” of most relevance for them. They can and should apply DSE strategies and concepts to their component in a manner equally valid as those in charge of the entire larger system. DSE’s concepts, practices, and terminology must allow for these differences of point of view and should enable accurate communication of information across them.

4.2. Value theory strategies DSE applies the following strategies related to value theory.

 DSE prefers to construct von Neumann-Morgenstern (vN-M) utility functions whenever possible.  When it is not possible to construct vN-M utility, other goals, constraints, or uses of the system can be used to define system goals and preferences.  Specification of requirements should be delayed as long as practicable during system design and development, in favor of negotiable preferences.

Von Neumann-Morgenstern utility functions4 were the starting point for the development of game theory, and is now the basis for an active thread of engineering research in what is often called value theory. This research is based on the idea that to make rational decisions from human preferences, one must create a metric of value that is based on a single axis of scalar numbers. For example, money measured in dollars, euros, or some other currency is a common way in which humans use a single scale of value across a variety of human preferences. Von Neumann and Morgenstern showed that if value can be measured with a single scalar metric, then this can be used as the basis for “rational decision.” This is a very strict interpretation of what “rational” means, clearly fitting the needs of mathematical and economic research. Engineering researchers are using these ideas as a means to rigorously specify the purpose(s) of a system, and then be able to assess designs against those purposes. Ideally, one desires to create and select the optimal design among all possible design options. For systems whose purposes can be represented monetarily, the application of vN-M utility is relatively straight- forward. However, for systems for which profit is not the primary purpose, then some other scalar metric could be selected as the single measure of value for that system. For systems that perform scientific functions or for human spaceflight, use of vN-M utility is often problematic, and is subject to wide variations of opinion. Mathematicians, scientists, and engineers have been down these kinds of paths before. The experiences of trying to quantify human values, and of applying SE to less engineering-centric applications should engender some caution among those applying value theory to engineering. Attempts to quantify human values so as to aid prediction of future events go all the way back to the 1800s at least, when the methods of probability were being explored and applied to a variety of areas, including courtroom arguments, demography, and games of chance. Some of these applications worked well, but others, most prominently those attempting to quantify human values and psychology, were largely abandoned and not taken up again until the 20th century.5 Early in the century, the United States Army Stephen B. Johnson 4

Corps of Engineers was required to quantify the costs and benefits of building dams, which had to account for environmental costs and benefits, as well as those on humans in the area. Given the vastly different priorities of environmentalists, farmers, ranchers, and city dwellers, these analyses provide excellent examples of the difficulties involved.6 This kind of analysis continues to spur furious debate. In the 1960s, SE’s successes inspired attempts to apply its methods to less technical applications such as city planning and social welfare. These attempts were less than shining successes, once again due to the vastly different values of the stakeholders, the difficulties of representing those values, and the evolving nature of the values, organizations, and technologies involved.7 This resulted in critiques of the entire enterprise of attempting to apply systems analysis and engineering methods to messy social applications.8 The urge to quantify is deeply rooted among mathematicians, scientists, engineers, and economists, and it works best when applied to those domains. Von Neumann and Morgenstern initially applied these ideas to games and to economics, with some success. Whether those same approaches can be successfully applied to accurately or even adequately represent non-monetizable human values sufficiently to enable the rigorous definition of a system’s goals (and from there to enable optimization) is at best an open question. Clearly some applications are far easier than others. Some have argued that the zeal for quantification is really a way to provide a gloss of objectivity to messy, subjective reality. When put forward quantitatively, otherwise obviously subjective arguments appear “objective.” As these last two paragraphs imply, this author has considerable skepticism regarding current attempts to make human values fully quantifiable for any and all engineered systems. Nonetheless, some features of this work appear valuable. Systems engineers should quantify values on a single scale whenever possible. Value theorists emphasize that what drives the system development is “preference” measured by value, and not “requirements” in the usual sense of the term. One way to understand this argument is to note that if engineers find that a requirement cannot be met, then instead of giving up on the system, what usually happens is a negotiation to alter requirements so as to achieve as much as can be achieved with available resources and schedules. What “really” exists are preferences to achieve some goals more than others, and not “hard requirements.” There is much merit to this point of view, but it remains true that eventually a real system must be built and operated. This as-built system has real limits on its capabilities, and those limits define real constraints on what goals can be achieved. Leading up to that as-built system, the negotiations of what should be built eventually turn into requirements on what is built. Exploring how best to prolong those negotiations before defining the actual requirements and constraints needed to build the system is an important objective of DSE. An example of this is the physical location of a launch tower with respect to a launch vehicle at that location. For the purpose of avoiding hitting the tower at launch (due, for example, to winds, launch vehicle and tower bending modes, and failures of various kinds), it is preferable to keep the launch vehicle far away from the tower. However, as a primary purpose of the tower is to provide access to the launch vehicle and its payload, from the standpoint of access it is better to keep the tower close to the launch vehicle so as to reduce the length (and hence mass and cost) of cabling, walkways, and such. One can describe these as “preferences” regarding the distance of the tower from the launch vehicle. From the standpoint of value theory, it is best not to write “requirements” that state “the launch vehicle must be positioned between 8 and 10 feet from the tower”, but rather to note the two preferences, and let the relative value of those two preferences guide the negotiations about the relative distance between the two. Eventually a decision must actually be made, a design based on that decision developed, and then the actual devices must be built. At this point of decision, preferences are replaced by requirements, which define constraints that cannot or should not be violated. In the real physical system, there are real physical constraints, and if these are violated, failure will ensue. In this tower-pad example, if the constraints are violated, a variety of failures can happen, including costly but not catastrophic problems such as cabling not reaching from tower to launcher, to major catastrophes such as the launcher hitting the tower causing a major explosion. Once the negotiations regarding proper distance are completed, the preferred distances become a required distance within tolerances.

4.3. Model strategies The following strategies derive from DSE’s model basis.

 DSE maximizes the use of models to represent, maintain, and generate knowledge.  DSE system-level representations shall at a minimum include those for value, optimization, intention, design, performance, behavior, and agency.  DSE shall provide abstract, system-level-compatible representations of discipline models. Stephen B. Johnson 5

Model-based SE (MBSE) is currently in vogue in SE research and development applications. DSE supports this emphasis, but only as a necessary, rather than sufficient, condition. It is necessary to specify what kinds of models are needed, and for what purposes, including the types listed in the second and third bullets. The reasons for the listed model types in bullet 2 are either obvious, or will become clear when describing the representations and the practices that use information from these representations. The meaning of “abstract system-level-compatible representations of discipline models” in bullet 3 is less obvious and deserves some attention here. A common feature of discipline models is that they represent some subset of attributes of the system and its components. Discipline models are the basis for CEDs to assess prospective performance of their parts of the system and thus to trade off different designs to achieve the goals levied on their disciplines. Ultimately, information from discipline models must be integrated into system-level models and assessments. To do this, it is useful to create abstract representations of the discipline models that integrate with the various system-level models, and can in some cases, when properly constructed, become system-level models. This enables tight integration of information from the disciplines to the system and vice versa. These simplified discipline models must be in representations that are “sufficiently abstract” to be used across, and not merely within disciplines. An example of this will be discussed in the “design models” section below.

4.4. State strategies The following strategies derive from DSE’s state basis.

 DSE shall maximize, to the extent practicable, the use of state variables in its representations.  DSE shall define functions as mappings of input states of state variables to output states of state variables, y = f(x).

As explained in the predecessor “Theoretical Foundations” paper, a technique common to CEDs is the use of state variables. In general, one can define the state variables that each CED “owns.” For example, electrical engineering owns state variables related to voltages, currents, electromagnetic field strength, and so on, while mechanical-structural engineering owns mass, stress, and strain state variables. DSE uses this commonality of state variable representation as a key part of the framework to interconnect information from CED representations to each other and to system-level representations. This interconnects the physical laws “owned” by each individual CED into a consistent, rigorous framework that enables system-level, as opposed to merely discipline-level analysis. Conversely, if state variables are not used, as is typical of TSE, then natural language is often used, with all the issues of communication that this entails. Another critical reason why state variables are so important is the need to distinguish between the “true state” of the world, and the “estimated state” that the system develops and “thinks it knows” about the world. “World” in this case refers to both the environment, and to the system itself. This is a crucial distinction in classical control theory, and by extension of these ideas, to DSE. Finally, by fiat in DSE, functions are defined as mappings of input states to output states. This cannot be done without the comprehensive use of state variables. Function is a major but often ill-defined concept in SE. It becomes a rigorously defined major concept in DSE.

4.5. Goal strategies The following strategies derive from DSE’s goal basis.

 To the maximum extent practicable, DSE shall define goals as constraints on the ranges of output state variables of a function over a specified period of time. Goal = rl < y < rh , where y = f(x) between times t0 and t1.  DSE shall represent system goals both in terms of operational description and of hierarchy.

The strategy in bullet 1 follows clearly as a rigorous way of defining goals in terms of previously-defined functions and state variables. It adds in the new concept of system constraints. A goal is said to be achieved if the state of the state variable is constrained within the relevant range within the relevant time period. The strategy in bullet 2 states the need for two distinct ways of representing what the system is intended to achieve. The first, “operational description,” is often called the Concept of Operations (ConOps). As goals are Stephen B. Johnson 6 rigorously defined, so too will be the ConOps that describes goals operationally. The ConOps, instead of being a natural-language description only, will be represented in formal mathematical-logical-physical goal statements. The need for hierarchy, which is explained in more depth in a prior paper on the Goal-Function Tree method,9 is related to the fact that engineered systems exist for one or more “end-purposes.” These end-purposes form the “top” of the tree hierarchy for the system. Other sub-goals exist to support the top-level goals, sub-sub-goals exist to support the sub-goals, and so on. The representation hierarchy exists because human intentions for the system are hierarchical in nature. DSE uses the “Goal-Function Tree” as its primary hierarchical model of intention.

4.6. Control strategies The following strategies derive from DSE’s control basis.

 DSE shall deploy passive and active means to control state variables within appropriate constraints so as to achieve the corresponding goal.  DSE shall use and extend classical control theory concepts of state estimation and state control to assess the system’s ability to achieve goals.  DSE shall simultaneously with nominal system design also address design of the system to mitigate the failure to achieve goals.

Because DSE conceives of engineered systems as mechanisms that use and control physical laws to achieve goals, DSE relies heavily on control theory concepts. That is, achieving a goal means constraining state variables within relevant ranges, which is what is meant by “controlling” the state variable. DSE takes it as axiomatic that engineering is fundamentally about control. Given this point of view, control theory concepts and strategies are essential. This does not mean that DSE is limited by current control theory. Rather, DSE assumes that current control theory applies, but also that its ideas must be extended. One major extension is the notion that passive mechanisms such as providing margins in design is also fundamentally about control.10 However, by definition, this is not “active” operationally, and classical control theory says little about it. Assessment of the performance of passive mechanisms (structural and thermal mechanisms being good examples) is a critical part of engineering, but it is not often thought of as “control” in the same way as classical feedback control. DSE finds value in representing the system in terms of both active and passive control. One of the key design decisions for any system is deciding whether active control, passive control, or some mixture of the two is best to achieve any particular goal.11 Control theory as it exists today is mainly about feedback control, and by its nature it generally assumes that the control system itself works properly to manipulate the “system under control.” However, there are many cases when the control system fails. Continuing with a launch vehicle example, the main part of the system to which classical control theory applies is the Guidance, Navigation, and Control (GN&C) subsystem. This uses measurements of acceleration and rotation that feed into GN&C algorithms, which in turn move rocket thrust (often through rocket engine gimbals) so as to keep the rocket accelerating in the proper direction. These algorithms work fine unless the system is disturbed beyond the GN&C subsystem’s capabilities. This can happen due to external events such as unexpectedly high winds, or to failures in the GN&C sensors, actuators, or computers. Dome launch vehicles have “fault management” capabilities to address the failure. These can bypass the failed components, or switch to new ones so that the regular control subsystem can again function properly. Sometimes the failure is so severe that the mission can no longer be achieved. In this case, the goals of the system can be changed to achieve some lesser goal such as not allowing the launcher to strike a city, to go to a different orbit, or in the case of a human launcher, for the crew to escape the failing vehicle. The fault management capability acts as a “meta- control system” to put the system into a state where it can once again be controlled, though perhaps to a lesser goal. Control theory concepts can be, and are being applied to the “fault management control loops” to assess their performance and assist with their design.12, 13 Another aspect of controls-style thinking is its use for assessment of performance. In classical control theory, one divides state estimation from state control. To achieve control it is generally necessary to know the state of the parts of the system that are being controlled. There are always discrepancies between the actual, true state of the system, and the perceived/estimated state of the system. This difference is the knowledge error. The limits of our knowledge of the system (the knowledge error), and the limitations to our abilities to control the system together define the limits of our abilities to control the system. Stephen B. Johnson 7

This kind of thinking can be extended to off-nominal situations. Failure is defined as the inability to achieve a goal, and as such is a problem of control to attain that goal. By contrast, the concept of “anomaly” is related to a problem of knowledge, in that the system behaves in ways that we neither expect nor understand. Systems can fail in perfectly predictable way, in which case we have failure without anomaly. An example is a car tire that over time wears out and goes flat. It is perfectly predictable that the material of the tire will eventually get so thin that it will break. We can also have anomalies without failure, such as an unexpected voltage profile in a test that is not so large as to be considered failure, but nonetheless does behave like the designers expected. Anomalies can sometimes be beneficial, such as when the Mars rover Spirit unexpectedly started generating more power than before. This turned out to be due to Martian dust devils (tornadic wind gusts) blowing dust off of the solar panels. The separation of the concept of anomaly from the concept of failure is crucial for the theory of System Health Management.12, 13

4.7. Knowledge strategies The following are strategies that derive from DSE’s knowledge basis.  DSE shall use existing sources of knowledge about the system.  DSE shall accept the variability of human interpretation of acceptable and expected system behaviors.  DSE shall maximize to the extent that is cost-effective the provision of automated means to cross-check the validity and consistency of information across all representations using different, overlapping knowledge sources.

Knowledge strategies address the human cognitive and social factors in the development of complex systems. These include the fact that while organizations generate and maintain knowledge, ultimately only humans “know” anything about the system. There is no collective mind, only individual minds in a collective enterprise. Working together, these individual minds can create a device that encapsulates their collective knowledge. One can consider the product or system as simply a vessel that encapsulates the knowledge of its designers and builders. The design and manufacture of complex systems generally involves multiple organizations, each of which groups its individuals in a host of sub-organizations. Each of these organizations is specialized to perform specific tasks, whether design or analysis of components or subsystems, testing, manufacture, operations, etc. The individuals within each organization in turn have specializations to enable their organization to achieve its purpose. These organizations can be viewed as repositories, generators, and maintainers of the knowledge required to complete their tasks or products. DSE accepts the existence of these organizations as the repositories of knowledge, and that DSE’s function is to coordinate and direct how the knowledge of the many organizations involved with a system are combined to create, build, and operate the system. Project management attempts to address the different methods by which individuals and organizations are organized to do this. Some organizations exist to support and propagate disciplinary knowledge. One set is academic institutions that educate and train individuals in the methods of the disciplines. Another set include so-called “functional” organizations in corporations and government institutions, often arranged in “matrix” form alongside the product- focused organizations. Disciplines are repositories of knowledge, and DSE must use and coordinate the knowledge coming from these disciplinary sources. One of the primary foci of DSE is to accept, coordinate, and address not merely the variety and types of knowledge, but the differences of outlook and outright incompatibilities of knowledge among the individuals and organizations. DSE accepts that different individuals and organizations have different interpretations of exactly the same (often natural language) requirement or system behavior. For example, in the lead-up to the 1986 Space Shuttle Challenger accident, O-ring charring phenomena were interpreted by some engineers as acceptable behavior that showed that the redundant O-ring design was working properly. Others engineers considered the charring as unacceptable behavior that should not be occurring and could not be fully explained. The same happened prior to the 2003 Columbia accident with respect to External Tank insulation foam falling off during ascent. These differences of opinion occur all the time. While one lesson for DSE is that some of the interpretations can lead to very bad results, another more fundamental one is that these differences are typical.14 Any theory of SE must accept the existence of these differences and have appropriate strategies to address them. Incompatibilities exist not merely in terms of interpretation of information but in the existence and consistency of information. Incompatibilities among the individual pieces of knowledge lead to the resulting device having incompatibilities. These can lead to the device behaving anomalously. Stephen B. Johnson 8

One major reason why a device behaves anomalously is the imperfection of human communication. Because complex system design is an inherently social activity (no individual knows enough to design and build the entire system), miscommunication between individuals leads to incorrect understanding of what the system should or can do. The resulting incompatibilities can lead to the system not achieving its goals. That is, the system fails.13 Systems also fail because individual humans cannot perform perfectly at all times. Attention flags. Simple calculation mistakes are made. Physical actions cannot be precisely repeated. While these mistakes are individual, finding the mistakes before they cause system failure typically requires social mechanisms. This is for the simple reason that individuals are not very good at perceiving their own mistakes. Other humans are much better at spotting them. In addition, cross-checks can be automated and implemented through testing. While these mechanisms appear “technical,” they are at heart “social” in that they use alternate knowledge sources to check the original knowledge embedded in the system. Automated checks are a major way in which DSE reduces the number of mistakes and incompatibilities within the system in a cost-effective manner.

4.8. Predictive strategies The following are strategies that derive from DSE’s predictive basis.  DSE shall use predictive models of performance, dependability, cost, and schedule.  DSE predictive models shall include assessments of uncertainty.

Making a “new future” come into being requires predictions about that future. This includes predictions about the projected nominal and off-nominal performance of the system within its anticipated environments, and about the mechanisms by which the system is created, built, and tested. The former is crucial to determine whether the system will achieve its goals, and the latter about whether the resources expended to achieve those goals are acceptable. If predictions about either differ from original intentions, then decisions may be required to modify the projected system or the resources to realize the projected system.

5. DSE representations DSE constructs and maintains a suite of models that represent various aspects of the system. One of the most important questions for DSE is determining the minimum number of models, or model types, necessary to perform DSE functions. In addition, DSE must address both success and failure (achievement of goals and non-achievement of goals). The hypothesized minimum suite of model types includes intention, design, behavior, physical containment, and institutions. The suite also includes failure space complements for some of them, for violation of intention, changes to design induced by failure, and changes to behavior induced by failure. It is assumed that the same institution set applies whether the system succeeds or fails. Finally, we assume that the suite of models for DSE must inherently connect to existing models, which generally are either disciplinary or product-focused terms. That is to say, existing models reflect disciplinary structures such as mechanical engineering, electrical engineering, control engineering, etc., or models that relate to the product that a given institution delivers, which often combines information from the disciplines within that organization. Figure 1 shows the proposed set of minimum model types, and the typical flow of information between these models. These models are arranged in a loop, indicating that models are developed and updated in a typical circular pattern. Over the life cycle of the system, multiple passages through the loop forms a spiral as models are progressively refined and eventually compared to the real system. We argue that engineering disciplines operate in just this way, with formal models built, refined, and used to inform their portion of the system design as well as assess its performance and help to verify and validate it. DSE is constructed and functions in the same way, but its model set integrates the various discipline models, which exist “outside” but nonetheless “inform” the system models shown in Figure 1. After a discussion about hierarchical and flat representations, the following subsections describe the major representation types, and the specific models within those types.

Stephen B. Johnson 9

Figure 1. DSE Models and Model Types in Closed Loop

5.1. Hierarchical versus flat representations There are two major classes of representations: hierarchical and flat. These two classes will be encountered frequently in the “model types” discussed in later portions of this section. An engineered system, unless its design structure really is a “tree,” does not have a hierarchical structure. Components connect to each other in a variety of ways, such as the loop structures common for feedback control. Components exist in many kinds of groupings like subsystems or volumes that contain other components. Component connections can change over time depending on the system’s different operating modes, and also based on failures. A simple example is a staged rocket, in which an entire stage is discarded. Failures sometimes create new and undesired connections, such as a short circuit creating a new electrical path, or an explosion creating patterns of debris that “connect” components. We call the manner of representation that enables components to connect to each other in the complex ways that exist in reality a “flat” representation. By contrast, a hierarchical representation inherently places some components or functions “over” others in a hierarchy. The existence of hierarchy implies an attribute that is given priority over others, enabling some functions or components in the fault tree to be “higher” in the hierarchy than others. One such attribute is intention, in which the system’s overall purpose, or (in a fault tree) failure of that purpose, is the highest-level node in the inverted tree structure. Physical containment models give priority to components that physically “surround” others as the hierarchical principle. Organizational hierarchy models give priority to authority and institutional control. In theory one can use almost any attribute as its organizing hierarchical principle. However, in practice only a few of these are common in SE; the three mentioned above are typical. Stephen B. Johnson 10

5.2. Value models Whenever requirement and design choices must be made, values are inherently involved. To ensure that choices are properly based on explicit values, and the values are addressed in a way that enables rational choice, a value model must be developed. In turn, this must be subject to the constraints of von Neumann and Morgenstern’s (vN- M) theory.15 Like other models of DSE, it becomes more detailed and refined over time, so as to enable rational, explicit choice at progressively more detailed levels of the design. Information produced from other DSE models frequently highlight issues or problems. These in turn imply the need for changes in intention, design, or operations, which the value model helps to judge or “rate” for effectiveness in meeting stakeholder preferences. DSE intends to use value models as far as is practicable as the system is iteratively developed.

5.3. Models of intention Every system has one or more purposes that must be defined. While value models prioritize preferences, these must ultimately be turned into specific intentions and goals for the system. In TSE, a ConOps provides a description of how the system is to be operated in its environment. Requirements and constraints are typically extracted from the ConOps by detailed reading and interpretation of this document. In DSE, it is expected that a written, natural-language description of system operations will often be generated. However, DSE adds rigor by developing an Event Sequence Model (ESM) to formally represent the operations described in the natural language ConOps. The ESM representation defines operations using state variables to provide descriptive precision and formality, and to ensure that the operations described in the ConOps document are assessed for potential gaps, overlaps, and contradictions. Major events in the ESM can be translated or extracted to define requirements that can be exported to a document if desired. The ESM is a contributor to the generation of a Goal-Function Tree (GFT) model (and vice versa), which provides a hierarchical representation of goals and functions in success space, associated with each other through the identification of state variables and constraints on the state variables. The GFT provides a precise functional decomposition of the system with its corresponding hierarchical system goals. This functional decomposition is physically accurate, and hence it can be used for a variety of analytical purposes. It inherently provides a top-down trace of goals (and when formally documented, requirements) in functional space. If desired, some or all of the goals can be exported from the GFT to generate a significant portion of a top-level system requirements document.9 Part of the modeling of intention is the modeling of system constraints, as these are necessary to translate preferences into goals. Some constraints are directly developed for specific goals and functions as part of the GFT and/or ConOps. However, not all constraints are attributes of specific functions. Some are system-level constraints that do not directly associate with a single function or function-related goal. These kinds of constraints are sometimes called “extensive attributes” that are associated with all or some system components, are “added up” with the number of components. These constraints include cost, mass, power, reliability, computer memory, computer processing speed, propellant mass, and the like. Some of these attributes are additive per component. An example is mass, of which each physical component has a certain amount, which added together generate the system mass. Not all attributes add in this way, such as system reliability or safety. When components are placed in parallel, system reliability and safety can improve even though more components are added. Costs add in a manner related to the number of components, but also are associated with operational tasks as well as physical components.

5.4. Design optimization Design optimization, or more generally, selection among design alternatives, is a critical function for design. In engineering research, “multi-disciplinary optimization” (MDO) is a well-known topic that addresses how best to accomplish this function. In DSE, design optimization uses the value model, ConOps, GFT, and various design and discipline models as inputs, from which to select the best design among the various alternatives. This can, does, and should happen at various levels of the design process, from the very earliest top-level design concepts to detailed design alternatives at lower levels and later parts of the design process. From the standpoint of DSE representations, design optimization uses a variety of models as shown in figure 1, but in general it cannot use or integrate them properly without modification. At its most basic, one can picture this as the use of existing system and subsystem models, with an added interface layer to enable connection of information Stephen B. Johnson 11 from that model to feed optimization algorithms. Depending on the information available from these existing models, they may need to be modified to create new information that is related, but not identical to what they already supply. This information, along with information from system preferences, costs, and other factors, feeds the optimization process, of which the outcome will be preferred integrated design solution for the problem at hand.

5.5. Models of design DSE requires high-level system design models. At least two types of design models are needed: directed graph and physical containment. Depending on the application domain, we expect there will be other types of relevance. The directed graph represents each component as a node, and an arc (arrow) between components for their interconnection. Loops are allowed, which makes the directed graph a non-hierarchical, flat representation. Directed graphs in success space and failure space are needed. Success-space directed graphs assume AND gates represent the default relationship of the inputs of each node, with OR gates representing redundancy. While directed graphs are normally associated with the linkage of electrical, data, and fluid-flow components, they can and should be used in a more general sense. One can consider three-dimensional analytic models such as those for structures, electromagnetic fields, and explosions as directed graphs with a finer, three-dimensional mesh. Thus the directed graph is an excellent candidate for the role of a common representation that is “abstract enough” to translate models that exist at a finer grain of detail into common system-level representations that can connect to each other.16 The physical containment model is a hierarchical representation in which components exist within the physical boundaries of other components. Thus for a staged chemical-propellant launch vehicle, the top level of the physical containment model separates the vehicle “segment” from the ground “segment.” Inside the vehicle segment are the first stage, boosters if applicable, second stage, and so on. Inside each of the stages are various components such as propellant tanks, propellant feedlines, electrical power and data wiring, computers, structures, propulsion, and so on. Inside of each of these are lower-level components such as valves, chips, wiring, etc. The physical containment model is crucial to connect organizations to designs.

5.6. Models of failure Failure models could potentially be grouped into two distinct groups of intention and design. Thus they could have been discussed in those sections. However, they have some unique features that make it worthwhile to consider them separately. A very common failure model is the fault tree, which can be constructed in part by taking the logical complement of the success-space GFT. In failure space, OR gates represent the default relationship of the inputs of each node, with AND gates representing redundancy. Because more links between components exist in failure space than success space, there are more tree branches in the fault tree than exist in a success tree like the GFT. A simple example of this is a short circuit, which creates an electrical path in failure space that does not exist in success space. This new path leads to a new branch in the fault tree that does not exist in the success-oriented GFT.9 Being hierarchical, a fault tree must use some attribute that is the basis for its hierarchical structure. Usually, fault trees are implicitly “intentional” in that they represent ways in which system goals fail to be achieved. The top branches of the fault tree represent failures of top-level goals. Like TSE functional decompositions, fault trees can have the same issues with use of natural language in that it is possible to model the system in “non-physical” ways. Fault trees that use probability numbers, such as in probabilistic risk assessment, are typically rigorous in the sense that the relationships between failures in the tree are arranged so that they are probabilistically correct. However, this does not guarantee that they are “causally correct” in the sense of the time-arrow of physical effect propagation through the system and its functions.9 Fault trees can be constructed independent of risk probabilities. This is typical of fault trees used for safety analyses, in which the purpose is to identify functions that cannot fail, and then determining proper “controls” to minimize the possibility of the function failing. Failure models can also use directed graphs and related representations such as Reliability Block Diagrams (RBD). These models are not hierarchical, and are constructed so that components are related to each other in ways that represent the actual component connectivity of the design. Thus these can be considered as a kind of design model. When modeling in failure space, directed graph design models typically represent the propagation of failure effects through the system. RBDs implicitly do the same. For example, when a set of redundant hardware is modeled in an RBD, the existence of a mechanism to “vote out” a bad string of components or ensure a good string Stephen B. Johnson 12 of components is used is assumed, even if not explicitly modeled. The modeling technique assumes that the failure effects generated by a bad string of components will not propagate beyond the (often implicit) voting mechanism.

5.7. Models of agency Organizations are the means, or “agents” by which the system is realized. Models related to these organizations are necessary. Most obviously, predictive models of cost and schedule to realize the system exist to ensure that the system can be realized within intended resources and time. Organization models enable the need to allocate requirements to the relevant organizations. One typical organizational model is the organization chart. It is typically hierarchical, but for different reasons than the GFT or other intention models. The reason that the organizational chart is hierarchical is the hierarchical structure of institutional control over the project or system. During the early phases of a system life cycle, usually one organization is in charge of the entire system. This organization typically hires other organizations to build the next level of components, and it integrates these components and contractors. These contractors hire subcontractors to build the next tier of components further down the managerial hierarchy, and so on until all components are allocated to organizations that build or purchase components. Organization models that mirror the organization chart are important because of the need to allocate requirements and ultimately funds to organizations to design, build, test, or operate their portions of the system. These allocations, at least in government organizations and contracts, are often implemented through Work Breakdown Structures and numbering systems.17 Another frequently-used type of organization model is the work process model. Work process models are not hierarchical, because work processes are usually sequences of processes or events, and can often be modeled with an Event Sequence Model. Schedule planning is based on the connection of work processes, and is typically managed with a variety of program management and scheduling tools based on event sequences. A simple example is the Gantt chart using a simple waterfall of events. More useful for coordination of large-scale complex projects is PERT (Program Evaluation and Review Technique), which was developed in the late 1950s and 1960s for military and government projects.18 PERT and its descendants are still used today, implemented in a variety of commercial tools. Aside from project management and scheduling tools, occasionally there are attempts to develop detailed work process models that go into great depth regarding the actual data being passed between groups. This kind of model, often associated with the development of software engineering “ontologies,” is much more detailed and difficult, and to date this sort of model remains a dream of what could be useful for SE as opposed to a standard technique of SE. It is possible that this kind of model could become generally useful and cost-effective, but not before much more rigorous modeling methods become standard practice. That is, implementation of something like DSE will be needed before there can be a cost-effective method of rigorous work process modeling and coordination beyond the scheduling techniques that have been typical of aerospace projects since the 1960s.

5.8. Models of behavior Along with cost and schedule criteria, the success or failure of a system depends on whether it achieves its goals. This means that its functions are properly performed, which in turn means that the system “behaves as it should.” It is no surprise then, that models that generate system behavior are a critical aspect of DSE. Behavioral models allow exploration of the state space implicit in the design, when driven by appropriate inputs to stimulate the many possible states of the system in both nominal and off-nominal scenarios. Exploration of the state space is necessary to ensure the design performs as intended, and more generally to verify the system. One major type of behavioral model is the state machine. State machine models generate complex behaviors from many simple models. That is, while many models require analysts to predict and model complex behaviors directly, state machine representations consist of many simple models of the state transitions of individual components, which can be triggered by the state transitions of other components, with relevant time delays as appropriate. The models are generally driven by an external script that defines how the system is operated. When environment models are added, and a state machine executive executes the component models, complex behaviors result. State machine models are useful for assessing the ways in which the system’s software and operational structures operate and interact with each other and the various hardware, software, and operational components. Being simpler and less expensive to run than “regular simulations or tests,” these models can be used to explore a much larger set of potential behaviors than regular testing can normally accomplish. These explorations, particularly when enhanced Stephen B. Johnson 13 by formal methods to comprehensively model the state combinations, provide a key means of generating and verifying system behavior analytically.19 Another critically important set of models that generate behaviors are those that drive simulations. These are generally physics-based models, which can mix combinations of models of the system and actual components of the system, up to and including the entire system, within a simulated environment. Driven by inputs that create nominal and off-nominal scenarios, simulations create huge amounts of data that must be assessed. Simulations are generally more expensive than state machine runs, both in cost and time. This is because they have greater fidelity to the actual system, though this can and does vary significantly both across systems and within systems during its life cycle. As simulations incorporate more elements of the actual system design, these transition to tests. When the entire system is operated in its real environment, we proceed into the realm of pure testing.

5.9. Models of performance Simulations are often used to assess system performance. However, other kinds of models also assess performance without generating system behavior. A classic example is the root locus analysis of control theory, which assesses the projected stability of the control system, without explicitly generating system behavior over time. Another example is models used to assess performance of Fault Management Control Loops, in which models are used to estimate performance. These are expressed in terms of fault management metrics that are inspired by, but not directly comparable to classical control theory.20 Some models exist to provide a structure in which to collect and relate empirical data in an analytical framework. Some of these models exist in the engineering disciplines, and some may exist at the system level.

5.10. Discipline models While discipline models are by definition not SE models and thus not DSE models either, information from disciplines must be integrated into DSE models. When considering the types and content of DSE representations, information coming from discipline models must be integrated and used in DSE models. In general this can be done because both discipline and DSE models typically use state variables in which their inputs and outputs are expressed. This enables mapping of discipline to DSE representations.

6. Using DSE models Engineering models are of no value unless they are used for constructive engineering purposes. As described in section 3, SE exists to optimize performance, to reduce the chance of failure, support cost and schedule prediction, and speed development. It does so by coordinating and integrating information from disciplines and organizations. TSE produces several key products, which DSE should be able to reproduce (at least in part) with greater accuracy and rigor, at lower cost, and in less time. Typical products of TSE include (but are not limited to) requirement traces and allocations, interface requirement and control documents, risk assessments (including reliability, safety, cost, and schedule), system performance reports including margins between existing performance and relevant constraints, and requirements verification and validation. This section describes how to use and operate on DSE models to produce these products, and the means to do these at lower cost and in less time than TSE.

6.1. Model and design spiral As depicted in Figure 2, TSE is frequently described in terms of a “V” diagram in which one starts at the left upper portion of the V with top-level needs. One then progressively moves to the right and down the V, defining requirements and designs until all details of the design are known. Various documents are produced along the way, particularly those defining the requirements and the interfaces between components. The system components are built at the bottom of the “V,” and then as these components are integrated, there is a progressive verification of the system from the smallest components, to integration of these components into units and testing that integration, and so on at higher and higher levels of the system. This process moves up the right side of the V. Verification in each of these steps refers to ensuring that the system’s components “are built right” or “performs their functions adequately with respect to requirements.” At the end of the V at the upper right, all components and integrations of components have been verified, and one next validates the system, determining whether one “built the right system,” or Stephen B. Johnson 14 alternately, whether the requirements were correct insofar as meeting the customer’s needs. Whereas verification is conceptually straight-forward because it tests the system’s behaviors and performance against known requirements that are drawn from the documents developed during design, validation is a much more difficult concept, as it calls into question the requirements. Traceability of requirements from top to bottom during design, from bottom to top during verification, and from the requirements documents to the verification of those requirements is central to the process.

Figure 2. TSE “V” from Concept of Operations through Validation

As one proceeds down the left side of the V during design, there is a back and forth process between definition of requirements and of designs that fulfill those requirements. Once a design is accomplished at one level, it is used as the basis for establishing requirements at the next, lower, more detailed level. This in turn drives the design at that level, and then to requirements at the next lower level, and so on. In practice, this process is not as neat and linear as the diagram implies, as the design and requirements processes intermix in parallel with each other. Additionally, significant issues that are discovered at lower, more detailed levels can lead to re-assessment of requirements or designs at higher levels. Problems found during V&V can also lead to design and requirement changes, and then retesting and re-analysis of the changes. Nonetheless, there is a fair amount of “truth” to this depiction of the integration aspects of TSE. How might we describe or replicate the functions being described in this TSE “V” in terms of DSE? First, DSE is based on a suite of models. Requirements exist as “models of intention” at the upper left of Figure 1. Designs appear in design models at left center. Verification is not directly indicated as a model, because it is a comparison of requirements that exist in the models of intention versus system behaviors. System behaviors are generated through the models on the lower right of the diagram. Some verification is also accomplished analytically through comparison of the outputs of the models of intention with performance models in the center of Figure 1. Validation, to the extent it relates to the original ConOps, is a comparison of the Event Sequence Model for the ConOps to the behaviors generated through behavior models and through actual tests of the system. The back and forth nature of requirements development and design, and later between component integration and verification is represented by moving “around the loop” in DSE and repeating operations. During design, the loop of model and design refinement is repeated to more detailed levels of design, forming a spiral. Imagine a spiral with its axis centered on the left side of Figure 2. Verification is also a spiral of sorts, though in this case between integration tests linked to requirements and the system components as they are integrated. In DSE, models of behavior are more Stephen B. Johnson 15 prominent at lower levels of verification, as system components are driven by simulations of the components they connect to. As more actual components are integrated, simulated components disappear until there is an actual complete system to test, ideally in its operational environment. As test results come in, the models of the relevant components are updated to match reality, and DSE operates to progressively refine models to more closely match the actual system and its behaviors. Improving DSE models is particularly important to aid continuing assessment of potential off-nominal behaviors, and to reflect updates to the system to achieve different goals or to lower costs.

6.2. Requirements development, allocation, and traceability One of the primary functions of SE is to develop requirements, allocate them to the relevant components and to the organizations that build them, and to trace these requirements from higher (more general, applying to system as a whole) to lower (more detailed, applying to system components) levels. DSE ideally starts this process with the development of value models. DSE “prefers” to delay the transition from preference to goals and requirements as must as practicable. It remains both a research and practical question to determine when it is appropriate to make this transition as a system is developed. It is a practical question in that regardless of what the research determines to be ideal, there are frequently legal and contractual mechanisms that exist, which cannot be ignored. The ConOps and the GFT are the system intentional representations. Both require state variables, but the constraints on them do not have to be defined right at the start. They can be filled in later as the design process proceeds. Models of intention define the top-level goals that a prospective design is to fulfill. The models of intention and of design drive each other progressively “deeper” to greater detail in a spiral process through the design optimization process. As it is a model of functions and goals, the GFT inherently shows the traceability of requirements from higher to lower levels in “function space” that does not directly represent the design. The most crucial sort of traceability tracks requirements from the system level to lower levels in terms of components and of the organizations that build these major components. Requirements must be levied on organizations, and thus a crucial issue is how to translate hierarchical goals modeled in “function space” to institutions that build system components. The process is to map goals and functions to designs, and then to organizations that build those designs. This can be done because of DSE’s use of state variables in the intention and design representations, and because the design can be associated with the organizations that implement it. The ConOps and GFT models contain the functions and goals, specified in terms of input and output state variables of functions, and then constraints on the output state variables. Since designs are simply mechanisms to achieve functions, the output state variables of a design mechanism must be the same as the outputs of the function that the mechanism is implementing. Mapping from the ConOps and GFT to the design is simply a matter of identifying the state variables and constraints in the intention models with the exact same state variables and constraints in the directed graph design models. Proper state variable and constraint naming conventions are crucial. Mapping to organizations requires the use of the physical containment model. This model contains state variables at the interfaces of the components that define the “containment regions.” One important reason that DSE uses the physical containment model is the strong correlation between the containment components and the organizations that build those components. It is generally true that a single organization is responsible for integrating all sub- components that exist inside a component. Thus the existence of sub-components inside a component hierarchically mirrors the authority structure of the organizations that build the component and sub-components. The traceability and allocation of requirements from top to bottom and from function to organizations therefore moves from intention models (ConOps and GFT) to design models by identification of state variables. The next mapping is from the directed graph design models to the physical containment model, once again through identification of the same state variables that exist both in the directed graph and physical containment models. Finally, the organization model closely mirrors the physical containment model. Both of these are hierarchical, allowing mapping from physical design components to the organizations that build these components. We thus have complete traceability from function to organizational hierarchy. With appropriate tools, this traceability enables auto-construction of key portions of major SE documents. These include the system requirements document, the component requirement documents, and interface control documents. Auto-construction of these documents can be accomplished by association of the relevant organizations to the goals mapped to them. Each goal is defined in the original models of intent, mapped to the appropriate organization, and then extracted to a document via a search for the organization and the goals mapped to them. Most Stephen B. Johnson 16 requirements can be extracted from the models in this way, but not all contents of the relevant documents can be generated like this, such as design images and schematics.

6.3. Failure and risk assessment The models of DSE encompass both nominal and off-nominal representations, and these models have important relationships to each other that must be exploited so as to decrease costs and increase consistency of the models. One example of this is the GFT, which can be used to form the starting point for system fault tree(s). Since every goal in the GFT are defined as constraints on the output state variables of functions over a certain time span, failure of the goal is simply defined as the state variable being outside of this range. Since the GFT is hierarchical, taking its logical complement creates a fault tree. Failure models inherently represent a larger number of behaviors than success-space models, because some failures create failure effect paths that do not exist in the nominal design. A full exploration of system behaviors thus requires failure-space models, not merely success-space models. The complete system fault tree is larger than the corresponding success tree because it has all of the GFT branches but adds new failure effect branches. The same holds true for directed graph models of the system design, and the failure space directed graph of that same design. The more complete representations for a given aspect of a system are in failure space, not success space. Once the fault tree is constructed by adding failure paths from the “starter fault tree” constructed from the logical complement of the GFT, the fault tree can be used to generate an initial functional Failure Modes and Effects Analysis (FMEA) and list of system failure scenarios. The bottom-most node of the tree that is assessed as “failed” is a failure mode for the FMEA, and the effect fields of the FMEA are the compromised functions and goals moving up the tree from a given failure mode. This becomes possible because the GFT, and hence its logical complement fault tree, are always constructed with “state variable rigor,” which ensures proper physical causality. Because causality is properly modeled, each path from the bottom or from the middle of the fault tree defines a failure scenario. These can be automatically extracted simply by searching and documenting all paths up the tree. These can later be used for system verification. Failure scenarios represent the suite of behaviors represented by a path from any node in the fault tree up the tree until it either meets a redundancy node or continues to the top. The same kind of relationship exists between the nominal design directed graph and its logical complement failure space directed graph that also adds failure paths. Failure scenarios are modeled by selecting a node in the failure space directed graph, and allowing the failure effects to propagate from node to node through the directed arcs that link them. In general, the directed graph set of failure scenarios should be compared with the hierarchical representations of the fault trees to determine if there are any differences between the two representations. If there are, one or both of the representations is likely to be deficient or erroneous in some way. Similarly, the success- space directed graph representing nominal connections between components can be compared to the GFT. Finally, the state machine models (discussed in the next section) should generate the same sequence of nominal or failure events through its bottom-up modeling technique. Once the failure scenarios are constructed, they can be analyzed through a variety of quantitative and qualitative techniques to determine the effectiveness of the detections and responses to failure. These in turn feed system-level quantitative estimates of meeting system dependability goals, as the performance of the fault management is one of the key metrics for system dependability. Another aspect of risk management is reporting and management of design margins. DSE is not fundamentally different from TSE in this regard, except that the data measuring various attributes that are being reported for margins are extracted from DSE models and then automatically mapped into and combined in a “margin model” that can be as simple as spreadsheets that are typically used for the purpose in TSE. These extensive attributes (which include things like mass or cost) frequently act as system-level constraints on the system, which can then be used to trade different design options for accomplishing system functions defined in the GFT, or to determine how best to control the system physics through a design. Performance measures associated with different design options can be assessed in traditional ways by disciplines, and then integrated in the GFT and extensive attribute / constraint models of DSE. Ideally, design options are assessed using choice models using VN-M utility. The output from these analyses lead to definition of specific and documented system requirements, in which system constraints are implemented by placing threshold values on the state variables of the GFT. These in turn are subdivided and Stephen B. Johnson 17 allocated to modeled system functions and from there mapped to physical design and then to organizations that implement them in the same way as requirements in general.

6.4. Verification and validation Within the model-based processes of DSE, verification and validation (V&V) uses a progressive refinement of the models based on actual integration and test results, along with exploration and analysis of the system state space that cannot be tested for all possible nominal and off-nominal states. Thus V&V is ultimately analytical, because testing typically cannot cover all possibilities. Test results are directly compared with model and analysis predictions to ensure that system and model states and behaviors correlate correctly, while flaws in the system design, manufacturing, and operations are corrected as anomalies are discovered and investigated. For every change made to the system, corresponding changes must be made to the appropriate models. One of the best types of model to search the off-nominal space is the state machine model. State machine models were developed and used to assess software, but DSE expands this to include the entire system and its states. State machine models are built from the bottom up, such that “the model” is really many simple models, each representing how a single components (these can be hardware, software, or humans) changes output state based on changes to input states. When “executed”, an output from an individual component is tracked by the execution software, which then searches for any other components that use the new output state as an input. For any component that does, then that component is “executed” to determine what output states change based on that input. This process is repeated as often as needed to yield the total set of system state changes across all components over time. Because the state machine model is a very simplified, all-software representation, it can be executed much faster and across many more kinds of situations than more realistic physics-based simulations or tests. State machine models create scenarios naturally, simply by inserting a failure behavior (which is a particular state) and allowing the model to execute to generate the resulting behaviors. Testing is generally based on scenarios, and as described above, the information to create these scenarios reside in the DSE models. Test scenarios can be extracted from the models and used as the basis for constructing test procedures. The information that is extractable from the models include the state variable and component state initial conditions, prediction of expected state changes during the test, and the expected result of the test. While it goes too far to say that DSE enables fully automatic generation of the test procedures, extraction of the information described above goes a long ways towards automation of test procedure generation. An interesting conjecture about the meaning of verification versus validation is implicit in the models of DSE. One possible interpretation of verification is that it relates to the testing and analysis of the system, when the results of these tests and analyses are compared to the requirements specified in the organizational hierarchy and documented as the requirements given to organizations and at the interfaces between organizations. Verification thus consists of documenting proper performance against requirements structured in the organization hierarchy tree. However, the original intentions for the system are stated in the GFT, which is a functional representation of system goals. Validation can be interpreted as the assessment of the test results from verification, when mapped into the GFT functional hierarchy, or the ConOps event sequence. Different kinds of tests are implied by the functional GFT structure, as compared to the organizational hierarchy structure. While verification works its way from the bottom up, validation refers the resulting information against the GFT from the top down, which in turn can be linked to the value model of stakeholder preferences.

6.5. Consistency checks, anomaly detection and investigation One of the major advantages of DSE over TSE is the possibility of automating consistency checks across the suite of DSE models so as to rapidly discover mistakes before they get into the actual system design. Because DSE models are mostly based on state variables, searches for the same state variable across many different models can enable consistency checks of various attributes associated with the components related to those same state variables. Cost-effectiveness of DSE will be greatly enhanced when tools to automate these checks exist. The contrast with trying to perform cross-checks on natural language documents is clear. Every scenario that is run in simulation and testing, and operations of the system itself in the real environment, generate reams of behavioral data. This data is analyzed using pattern recognition and anomaly detection, and any discrepancies or anomalies discovered are then assessed to determine their cause. These investigations determine Stephen B. Johnson 18 whether the anomalies are actual problems, or merely unexpected and not-understood behaviors that are nonetheless acceptable. Information from these investigations and from other ongoing analyses and assessments feed back into the decision processes to determine whether the system’s designs, goals, or models should be updated. “Big data” methods are becoming more prevalent to detect patterns and anomalies, operating on vast amounts of data in various applications. These include provision of buying and advertising recommendations by online sellers and service providers such as Amazon and Google, and searches for nefarious behavioral patterns of terrorist or criminal networks by law enforcement and intelligence agencies. In System Health Management, “anomaly detection” algorithms have been developed to identify unusual or unexpected data patterns. Neural networks can be trained to do the same thing. These pattern recognition and anomaly detection methods can and should be used to operate on the vast quantities of data that can be and are produced in state machine and simulation runs, and from operation of the as-built system(s). While these methods are good at identifying patterns, they are generally not as useful for identifying the significance of the identified anomalous patterns. To understand their significance typically requires human investigation to interpret the patterns and discern their underlying causes. For the purposes of DSE, these investigations as to the causes of pattern anomalies, along with any other problems discovered in the modeling and analysis processes described above provide information needed to fix problems with the design or projected operations with that design. For particularly difficult issues, it may be necessary to revisit old, or assess new, design choices available to address the problems discovered in the modeling and analysis process. Once choices must be made, then DSE requires the use of a value model to assess design options, which takes us back to the start of another loop in the DSE process.

7. Applying DSE The totality of the theory, representations, and processes of DSE described in this paper and its predecessor are new, and with some exceptions described next, untested. However, many of the ideas, sub-theories (such as control theory and aspects of systems theory), model types, and models themselves exist and have been used successfully in other contexts. The point of the theory of DSE, and of this paper, is to provide a foundation upon which the necessary research and development can be performed, so as to replace TSE with a far more rigorous, less error- prone, and less costly suite of methods. While this sometimes entails new model types, such as physical containment and GFTs, frequently this means interconnecting existing model types in new ways, which in turn often requires addition of new kinds of information into these models. Some portions of the theory and some of its models have been successfully implemented. The most prominent of these is the GFT, which has been developed and used on the National Aeronautics and Space Administration’s (NASA) Space Launch System (SLS) Program. On SLS, the GFT was used to identify failure conditions from which the crew needs to abort. Directed graph models are also being used on SLS to develop models of failure propagation, which are used to assess failure detection coverage, to trace failure modes to effects, and in the future to provide a diagnostic capability. A state machine model is also being developed to aid in the assessment of system command sequencing and to search for potentially hazardous system behaviors. Obviously state machine models have a long history, and directed graph models are now in use commercially for diagnostic systems. Other aspects of this theory have been identified in many other publications as crucial to the future of SE. Prominent among them is the need for MBSE, though there is much disagreement about what models need to be developed, and for what purposes. The theory of DSE in this paper attempts to provide much more specific guidance about the nature of those models, and the relationships required between them to produce useful results. The theory and the modeling approach expressed here are in development through ongoing research programs with NASA. We expect successful demonstration of more of these ideas over the next several years.

8. Conclusion This paper, the second of two that define the motivation, theoretical foundations, representations, and processes for DSE, focused on the goals, strategies, representations, and uses of the representations to achieve the goals of DSE. Starting with the bases and principles defined in from Part 1 of this two-paper set, this paper defined the goals and strategies of the new approach. DSE attempts to achieve at least the same goals as TSE, but with greater rigor, lower cost, and reduced schedule. To do this, it uses strategies derived from the “bases” of Part 1. Each major Stephen B. Johnson 19 component of the theory (that is, each “basis”), such as state, model, control, knowledge, etc., must lead to some useful approach that contributes to the new ways of performing SE that define DSE. DSE uses a specific set of representations, each of which performs a specific function necessary for SE. The value model balances the desires and preferences of stakeholders. Models of intention translate the balance of preferences into prioritized goals for the system. Design models define how the physics of the world is utilized to achieve those goals, in which the designs themselves are selected through an optimization process. Failure models describe the many ways in which the system can “not” achieve those goals. In general there are far more ways for a system to fail than to succeed, so the failure models are the most complete system representations. Agency models connect the design to those institutions that create, build, test, and operate the system and its components. Behavioral models generate system behaviors from the design, and performance models include some ways to assess performance that are not directly behavioral. All of these models must be connected to each other and to subsystem and discipline models so as to coordinate and connect the information about system components to each other to achieve the system’s goals. Finally, the paper describes how to use the suite of models to generate information needed by the designers, builders, testers, and operators of the system. For DSE, these uses must at a minimum reproduce the information produced by TSE, but with greater rigor and completeness, with less cost, and in less time. In addition, it should create new knowledge and products that TSE either has difficulty producing, or does not produce. This paper indicates at a top level how to generate these products, which include requirement traces, requirement allocation and decomposition, requirement verification, interface requirement and control documents, margin reports, risk reports, safety and reliability assessments, and FMEAs.

Acknowledgements The concepts described in this paper have further extended ideas from the theory of System Health Management developed by the author and also from John C. Day of Jet Propulsion Laboratory, largely from 2007-2011. As SHM is considered to be the “dark side” of SE, many of the ideas for SHM turn out to apply to “nominal” SE. Since 2012, several people have contributed important ideas and acted as productive sounding boards for the ideas described here. These include Bob Rasmussen at Jet Propulsion Laboratory who showed the importance of control theory and of “flat” representations, Jonathan Breckenridge at NASA Marshall Space Flight Center helped develop practical implementation of the Goal-Function Tree methodology, and Bob Garrett with Missile Defense Agency, whose ideas helped extend the “closed loop” of the representations. Some of the ideas in this paper were developed in part with a contract from the U.S. Army Aviation and Missile Research, Development, and Engineering Center (AMRDEC) to the University of Colorado, Colorado Springs on System of Systems Engineering research in 2014. Of particular importance has been the SE research consortium led by Dr. Michael D. Watson of MSFC through a contract from NASA to the University of Alabama in Huntsville (UAH) since 2012. Dr. Phil Farrington has been the Principal Investigator at UAH. This contract, prime contract NNM11AA01A, subcontract SUB2012-037 has funded the author’s systems engineering research at UCCS since 2012. Within that consortium, Paul Collopy from UAH has enlightened me regarding the notion of extensive attributes, and he and Rich Malak from Texas A&M persuaded me of the necessity of value models; we continue to productively debate values, preferences, and requirements.

References 1. Johnson SB, and Day JC. Theoretical foundations for the discipline of systems engineering. AIAA Infotech Conference; San Diego: AIAA; 4-8 January 2016. 2. Johnson SB. Three approaches to big technology: operations research, systems engineering, and project management. Technology and Culture 1997; 38:4: 891-919. 3. Johnson SB. The secret of apollo: systems management in american and european space programs. Baltimore: Johns Hopkins University Press; 2002. 4. Von Neumann J, and Morgenstern O. The theory of games and economic behavior. 3rd ed., Princeton: Princeton University Press; 1953 5. Daston L. Classical probability in the enlightenment. Princeton: Princeton University Press; 1988. 6. Porter TM. Trust in numbers: the pursuit of objectivity in science and public life. Princeton: Princeton University Press; 1996. 7. Jardini DR. Out of the blue yonder: the transfer of systems thinking from the pentagon to the great society, 1961-1965. In: Hughes AC, Hughes TP editors. Systems, experts, and computers: the systems approach in management and engineering, world war ii and after. Cambridge, Massachusetts: MIT Press; 2000. p. 311-57. 8. Hoos IR. Systems analysis in public policy: a critique. revised ed. Berkeley: University of California Press; 1983. Stephen B. Johnson 20

9. Johnson SB. Goal-function tree modeling for systems engineering and fault management. AIAA Infotech@Aerospace (I@A) Conference 2013. Paper 2013-4576. Boston, MA: AIAA; August 19-22, 2013. 10. Austin-Breneman, J., Yu, B. Y., Yang, M. C., Biased information passing between subsystems over time in complex system design, Proceedings of the ASME 2014 International Design Engineering Technical Conference & Computers and Information in Engineering Conference, Buffalo, NY, August 2014. 11. Day JC, and Johnson SB. Fault management design strategies. Space Operations 2014 Conference; Pasadena, California: AIAA; May 2014. 12. Johnson SB, Day JC. Conceptual framework for a fault management design methodology. AIAA Infotech Conference, Atlanta, Georgia: AIAA-2010-227006; April 2010. 13. Johnson SB, Gormley TJ, Kessler SS, Mott C, Patterson-Hine A, Reichard KM, Scandura, Jr. PA, eds. System health management: with aerospace applications. Chichester, United Kingdom: John Wiley United Kingdom; 2011. 14. Johnson SB, From the secret of apollo to the lessons of failure: the uses and abuses of systems engineering and project management at NASA, in Dick SJ, ed., NASA’s First 50 Years, A Historical Perspective, NASA SP-2010-4704. Washington, DC: NASA; 2010, Chapter 12, pp. 287-324. 15. Hazelrigg GA. A framework for decision-based engineering design”, Journal of Mechanical Design December 1998; 120: 653-8. 16. Garrett, Jr. RK. A graph-based metamodel for enabling system of systems engineering. Schriever, Colorado: Missile Defense Agency 14- MDA-7691; 19 February 2014. 17. Blanchard BS, Fabrycky WJ, Systems engineering and analysis. New York: Prentice-Hall, Inc.; 1981. 18. Johnson SB. Technical and institutional factors in the emergence of project management. International Journal of Project Management 2013; 31: 670-81 19. Berg P, Johnson SB, Trevino L. State analysis modeling for space launch system mission and fault management analysis. AIAA Infotech Conference 2016; San Diego, California: AIAA, January 4-8, 2016. 20. Lo Y, Johnson SB, Breckenridge J, Application of fault management theory to the quantitative selection of a launch vehicle abort trigger suite. IEEE 2014 Prognostics and Health Management Conference, Spokane, Washington; IEEE, June 2014.