© Copyright AAMI 2013. Single user license only. Copying, networking, and distribution prohibited. Features PERSPECTIVE The Drawbacks in Using The Term ‘System of Systems’ Nancy Leveson A few years ago, a new term, “system of structures. A model of a complex system can About the Author systems,” was invented and has become quite be conceived in terms of a hierarchy of levels Nancy Leveson, popular. I’ve puzzled over this term because of organization, each more complex than the PhD, is a professor it doesn’t make any sense to me with respect one below. Each level of the hierarchy can be of aeronautics and to systems theory and systems engineering. thought of as a system, which is made up of astronautics and Let’s start by reviewing some basic defini- components at a lower level. Each of these engineering systems tions in systems theory. components (or subsystems) can itself be at Massachussetts Institute of A system can be defined as a set of compo- made up of subsystems, and so on. Figure 1 Technology, and is the author of nents that act together as a whole to achieve shows a depiction of a system labeled A (level several books on system concepts. some common goal, objective, or end. The 1 of the hierarchy) composed of three subsys- E-mail: [email protected] components are all interrelated and are either tems A1, A2, and A3 at level 2 of the directly or indirectly connected to each other. hierarchy, each of which is made up of other The system state at any point in time is the set components (level 3 of the hierarchy). Note of relevant properties describing the system at that the term “system” is recursive in that a that time. The system environment is a set of subsystem is itself a system, which is made components (and their properties) that are not part of the system, but whose behavior can affect the system state. The existence of a boundary between the system and its environ- A Level 1 ment implicitly defines inputs or outputs as anything that crosses that boundary. It is important to understand that a system is always a model—an abstraction conceived by the viewer of the system. Systems and A1 A3 their boundaries do not exist in reality but only in the view of the beholder. One viewer A2 may see a very different system than another Level 2 in terms of where the boundaries are drawn, the relevant system properties and compo- nents, and even the purpose of the system. Level 3 Abstractions are useful in that they help humans deal with complexity. One useful abstraction in understanding complex Figure 1. System A is composed of three subsystems A1, A2, and A3. Each of these systems is to view them as hierarchical subsystems may themselves be composed of other subsystems (components). Biomedical Instrumentation & Technology March/April 2013 115 © Copyright AAMI 2013. Single user license only. Copying, networking, and distribution prohibited. Features up of subsystems and so on. The difference that a “system of systems” is made up of is only at what level of the hierarchy (“granu- already existing systems. But almost all larity”) the system is currently being viewed. systems are made up of existing subsystems. The subsystems A1, A2, and A3, when When creating a new system, rarely does viewed by themselves, is each a “system” with anyone create everything from scratch, down its own subsystems. to the screws and bolts. But even if they did, A system can also be viewed as part of a it does not negate the second basic concept in larger system. Figure 2 views system A as systems theory, which is emergence. part (a subsystem or component) of a larger Each level of a system hierarchy is character- system, AB, which has ized by having emergent properties. The two components or concept of emergence is that, at any level of I’ve heard people claim that the subsystems A and B. complexity, some properties characteristic of difference is that a “system of systems” There is no difference that level (emergent at that level) are irreduc- between considering AB ible. They arise through interactions among is made up of already existing systems. as a system with compo- the components at a lower level of complexity But almost all systems are made up of nents (subsystems) (a lower level of the hierarchy). Such proper- existing subsystems. labeled A and B, or as a ties do not exist at the lower levels in the sense “system of systems” or a that they are meaningless in the language “system of subsystems” appropriate to those levels. For example, the or whatever other term one wants to invent. emergent property of the shape of an apple, All these terms are identical in what they although eventually explainable in terms of represent and there is no need for a new the cells of the apple, has no meaning at the term that seems to imply that it is a different individual cell level. As another example, thing and can or must be treated differently. consider the property of gridlock in traffic. I’ve heard people claim that the difference is Looking at an individual car, the concept of AB Level 1 B A Level 2 B1 B2 A1 A3 A2 Level 3 Level 4 Figure 2. System A is here viewed as a component (subsystem) of a larger system AB 116 Biomedical Instrumentation & Technology March/April 2013 © Copyright AAMI 2013. Single user license only. Copying, networking, and distribution prohibited. Features gridlock has no meaning. Gridlock as a hospital patient might be a nurse connecting property emerges only when the highway the wrong lines together, for example, system is viewed as a larger system where connecting a feeding tube to an intravenous many cars, along with a particular design of a tube.* When considering only the intrave- roadway and other components of the high- nous feeding system, this hazard does not way system and its environment, interact. arise and, in fact, is not visible. It emerges Emergent properties arise from the interac- only when all the lines into a patient are tion of lower level components in the considered. System hazards exist only at the hierarchical system structure. system level, although it is usually necessary Now what does all this have to do with to inspect the design of the individual system safety of medical devices? Safety is an components to identify potential causes of emergent property. It is possible for individ- the system hazards. ual system components to have hazards, for At the 2012 AAMI/FDA Interoperability example sharp edges, flammable parts, or Summit, I was surprised at the number of unsafe individual operation such as an presentations that seemed to assume that infusion pump overdosing a patient. These safety analysis can be performed on individual hazards usually are not affected by putting components and then the components can be these individual systems together into a put together into a system that will be safe. system, although they could be. Because safety is an emergent property, this But other hazards arise only when compo- assumption violates the most basic concepts nents are considered together within a larger in systems theory and systems engineering. system where they interact either directly or This is when talking about “systems of indirectly. Analyzing the safety of only one systems” becomes dangerous because it individual component of that system does not somehow assumes that a “system of systems” and cannot consider the emergent safety is different than a system. It is not. The terms problems that arise when putting two or more “system” and “system of systems” have the components together. Usually, the hazards exact same meaning and the same top-down that need to be considered at the system level system engineering techniques have to be are different than those at the component applied. Bottom-up approaches cannot be level, but even if the hazards are the same, the causes are very different as the role of the The potential hazard of alarm overload can be associated interaction of the components comes into play as a potential cause of the hazard. with a single medical device, but the problem arises in a Let’s consider some simple examples. The different way when multiple devices, all with alarms, can potential hazard of alarm overload can be sound at the same time or interfere with each other. associated with a single medical device, but the problem arises in a different way when multiple devices, all with alarms, can sound used to analyze or assure safety in a complex at the same time or interfere with each other. system, even if one calls it a “system of The system-level problem of alarm overload systems.” Specifically, doing independent requires more than simply looking at an hazard analyses on individual components individual medical device or even several and then assuming those analyses can be devices. It requires looking at all the devices combined in some way to handle system that can sound alarms as well as considering hazards will not be effective. the characteristics of the system components Consider an aerospace example,† this time (probably humans) that must respond to the where the components interact with each alarms and any ways that one alarm might other. Suppose a flaps control system com- interfere with another. As another simple municates information to another aircraft medical example, a system-level hazard for a system that uses the information provided by * It is surprising how often this occurs even though simple techniques to eliminate the problem were identified decades ago by the aircraft industry to eliminate wiring errors.