1

On Complex Adaptive System (CASE) Brian E. White, Ph.D. Director, Process Office (SEPO) The MITRE Corporation

12 May 2008

8th Understanding Complex Systems Symposium University of Illinois at Champaign-Urbana 12-15 May 2008

Public Release Case Number: 08-1459

© 2008 The MITRE Corporation. All rights reserved Fundamentals of Enterprise Systems Engineering (ESE)* Outline

 Big Ideas  Basics of ESE – Making it an engineering discipline  Next generation systems thinking  An example

______* ESE can be thought of as the same as complex systems engineering (CSE) 2 [White, 2008b] © 2008 The MITRE Corporation. All rights reserved but there can be nuances of difference. See Charts 8 and 44. Big Ideas

 Complex systems abound – Mega-projects in transportation, the environment, U.S. DoD’s Global Information Grid (GIG), etc. – Internet culture—massive connectivity and interdependence  Complexity theory applies – Much activity in complexity science across many fields – University interest in developing ideas for engineering (MIT, Johns Hopkins, UCSD, USC, Stevens, UVM, U of I, Old Dominion)  Complexity is embedded in everyday knowledge – The Gardener metaphor (vs. The Watchmaker) – Biology and natural evolutionary processes – The way we think, our language/semantics – Markets (viz., The Wisdom of Crowds, The Black Swan)  Traditional Systems Engineering (TSE) methods may not help – But temptation is strong to keep trying them  One can dependably, but not predictably, build complex systems – Using systems thinking and Complex Systems Engineering (CSE)

3

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved A Spectrum of Systems See Notes Page

System: An instance of a set of degrees of freedom* having relationships with one another sufficiently cohesive to distinguish the system from its environment.**

*Normally grouped into subsets or elements **This cohesion is also called system identity

Less complex More complex Pre-specified Evolving

4 [White, 2008b] and [Kuras-White, 2005] © 2008 The MITRE Corporation. All rights reserved Relative Importance of Opportunity See Notes Page Uncertainty Risk Opportunity Un-assessable Unknown

Enterprise View

System of Systems View

Systems View

Risk Opportunity 5 [White, 2008b], [White, 2007b], [White, 2007a], and [White, 2006] © 2008 The MITRE Corporation. All rights reserved {View} = {Scope, Granularity, Mindset, Timeframe}

Large Enterprise

A change in a mind’s focus Inaccessible Region results in a change of View! (where a given human cannot conceptualize

No View change can take Scope one beyond this limit! SoS (e.g., FoV)

Time Accessible Region (where that human System Mindset can conceptualize) (e.g., cognitive focus) Small Coarse Fine Granularity (e.g., resolution)

6 [White, 2008b] and [White, 2007] © 2008 The MITRE Corporation. All rights reserved The Enneagram Web*

9/0 Identity Self-Organization Domains

Structure/Context 8 1 Intention Living Systems Command and Control Patterns and Processes Pattern and Process 2 Issues Learning 7 “Numerology” Coincidences? Octet: 0-7 order is logical Whole circle is “one” Living beings order: 1/7 = 0.1428571…

3 Relationships Information 6

The Work 5 4 Principles & Standards

______7 [White, 2008b] * [Knowles, 2002, pp. 30, 32, 33, and 39] © 2008 The MITRE Corporation. All rights reserved Degree of Difficulty View of Engineering See Notes Page Disciplines Systems Science Context Complexity Systems Theory Thinking System ESE Mission System Behavior Environment Strategic Context Context CSE′ Increasing Degree of Difficulty SE′ Desired Scope Outcome of Effort

Engineering′ Fundamental Applied Research Research

Stakeholder Scale Relationships of Effort Implementation Stakeholder Context Context Stakeholder Acquisition Involvement Modeling & Environment Simulation

8

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved Outline

 Big Ideas  Basics of ESE – Making it an engineering discipline  Next generation systems thinking  An example

9

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved Basics of Enterprise Systems Engineering

 An Enterprise is made up of a complex interaction of autonomous agents (people), processes, and technology; enterprises – Are recursively defined – Have transaction variables – Have multiple scales – Are socio-cultural, techno-economic- …  A process orientation is essential to ESE (see Chart 17) – Traditional Systems Engineering (TSE) is the “Inside Loop” – Enterprise Systems Engineering (ESE) is the “Outside Loop”  There are complementary mindsets and theories to be applied to ESE  Enterprise development involves several holistic areas/activities, e.g., – Balance between variety and uniformity – Shaping of interactions – Deciding what is nourished and what is not

10

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved Enterprise Definition

People, processes, and technology interacting with each other and their environment to achieve goals

Environmental Stress

people processes

technology

Enterprise Model

11

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved Recursive Nature of the Definition At every level an enterprise itself is a part of a larger enterprise and contains sub-enterprises of people, processes and technology

Public education

University

College

The sub-enterprise contributes to the goals of the larger enterprise.

12

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved Boundary for ESE—An Operational Definition See Notes Page

For the purposes of ESE, the enterprise is defined as the set of system elements that participating actors either control or influence. The remainder constitutes the enterprise environment.

Uncontrolled Environment Elements ESE Boundary Complementary ESE Processes Influenced TSE Processes Elements

Controlled Elements

The Enterprise boundary is a virtual construct depending on the make-up and authority of the participating actors and stakeholders.

13

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved ESE and Business Process Relationships

Throughput Process ESE Realm

Systems Development Process Create & Disseminate Systems/Services

Membership Leadership (Roles & (Vision, Goals Responsibility) Plan & Govern)

Learning & Conflict Control Management (Evaluate & (Coop, Compet., Assess) Trades)

Business Processes

14 [White, 2008b] and [Gharajedaghi, 1999] © 2008 The MITRE Corporation. All rights reserved A Challenge of Enterprise Governance

Traditional Enterprise Governance Governance Benefit to the Individual

Benefit to the Enterprise

Enterprise Governance Must Reverse the Traditional Rewards Curve

15

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved SE Process Moving to ESE

Integration Integration Influence or shape

Integrated Integrated & & Unchanging Complex

Selection Unorganized Unorganized & & Variation Unchanging Complex

Innovation (Differentiation) Innovation (Differentiation)

A “realized” enterprise re-invents itself through a The ESE process must shape the integration and process of continual innovation and integration.* innovation environment.** ______* After [Gharajedaghi, 1999] ** After [Axelrod-Cohen, 2001]

16

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved ESE Processes in Context

Gharajedaghi Enterprise Traditional Business Systems Engineering Systems Engineering Processes Processes Processes

Enterprise Management

Req Dev & Mgt Vision Strategic Technical Plan Risk Mgt Goals Config Mgt Conflict Mgt Cap Planning Analysis Tech Project Plng Technology Planning Roles & Resp QA System Safety ILS Shape Innovate Integrate

Restructure Process Design The Improvement Implement Enterprise (CMMI) Transition

Diagnostics Analysis & Assessment Integrated Test

Enterprise Assessment 17

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved Outline

 Big Ideas  Basics of ESE – Making it an engineering discipline  Next generation systems thinking  An example

18

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved Next Generation Systems Thinking*  Complex adaptive systems are … unique and non-reproducible, open, dynamic, self-organizing, unpredictable and surprising, sub-optimum but robust, unstable, …  Focus on – The big picture where the parts are characteristic of the whole – Engineering the environment of a complex system  Reductionism/Constructionism does not work when one cannot – Control stakeholders – Predict outcomes – Successfully prespecify future capabilities  ESE becomes more like a regimen** – Identify Outcome Spaces  Capabilities-based – Shape the developmental environment  Establish rewards (and penalties)  Distribute rewards for achievement of goals  Apply developmental stimulants (stir the pot and change the rules) – Couple development to operations  Characterize continuously (as in markets)  Enforce safety regulations that ensure continuous operation ______19 [White, 2008b] * See [White, 2008] for recent essays on complexity, complex systems and CSE ** [Kuras-White, 2005] © 2008 The MITRE Corporation. All rights reserved Enterprise Systems Engineering (ESE) Profiler

20 [White, 2008b] Source: Renee Stevens © 2008 The MITRE Corporation. All rights reserved Version 1 – 2 Oct 07 Systems Engineering Activity (SEA) Profiler

Typical Systems Left End of Left Intermediate Interval Center Intermediate Right Intermediate Right End of Engineering Activity Slider Interval Interval Slider

Define the System Establish Adapt to Changing Revise and Restate Try to Predict Future Discover Needed Problem System Requirements; Re-Scope Objectives Enterprise Needs Mission Requirements Capabilities Consider Technical Employ Research, Track, & Plan Research and Evaluate Pro-Actively Plan for Explore New Approaches Available for New Technologies New Technical Ideas Promising Techniques Techniques and Techniques Innovate Utilize a Guiding Apply an Develop Architectural Really Define (Not Just Adapt Architecture to Embrace an Architecture Existing Perspectives (Views) Views of) Architecture Accommodate Change Evolutionary Framework Architecture Analyze Alternatives Conduct Model/Simulate System Perform Systematic Include Social and Emphasize Systems Functionalities Cost-Benefit Analyses Psychological Factors Enterprise Tradeoffs Aspects Pursue Solutions Advocate One Consider Alternative Investigate Departures Iterate and Shape Keep Options System Solution Approaches from Planned Track Solution Space Open While Approach Evolving Answer Manage Emphasize and Mitigate System Risks Sort, Balance and Pursue Enterprise Prepare for Contingencies Manage System and Watch Opportunities Manage All Uncertainties Opportunities Unknown Risks Unknowns Develop Hatch System Prepare Enhancements Experiment in Develop in Realistic Innovate With Implementations Improvements for Fielding Operational Exercises Environments Users Safely Off-Line Integrate Operational Test and Work Towards Better Advance Horizontal Advocate for Needed Consolidate Capabilities Incorporate Interoperability Integration As Feasible Policy Changes Mission Functionalities Successes Learn by Evaluating Analyze and Fix Propose Operational Collect Value Metrics and Adjust Enterprise Promulgate Effectiveness Operational Effectiveness Measures Learn Lessons Approach Enterprise Problems Learning

Convenient Labels Traditional Systems Complex Systems (Only; interpret them): Engineering (TSE) Engineering (CSE) 21 [White, 2008b] Aggregate Assessment Source: Brian White of Above Slider Positions © 2008 The MITRE Corporation. All rights reserved Complex Adaptive Systems Engineering See Notes Page

 Conventional SE is insufficient and sometimes counterproductive, in addressing the most difficult SE problems.  As an alternative that may work better, we offer a Complex Adaptive Systems Engineering (CASE) methodology. – Create Climate for Change: Create a climate for engineering the environment of the System. Continually plan for agile, constructive change (accelerating the processes of natural evolution) through proactive dialog with stakeholders, especially customers. – Architect a Strategy: For the System, within its various system, system of systems (SoS), enterprise, and/or complex system contexts. – Target Outcome Spaces: Describe the customer’s mission/vision in terms of one or more desired outcome spaces, not solutions. – Reward Results: Work with the customer and a governing body to create appropriate incentives.

22

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved Complex Adaptive Systems Engineering See Notes Page (Concluded)

 CASE (concluded) – Formulate Decision-Making Heuristics: Discover and promulgate management heuristics that will help the customer better know how and when to make decisions. – Stimulate Natural Processes: Continually “stir the pot” by introducing variation (innovation) and selection (integration) while shaping and enabling future constructive change, and trying to avoid chaos and stasis, respectively. – Develop in Operational Regimes: Create a bias for developing evolutionary improvements of the System in actual operational environments with real users. – Assess, Learn, and Re-Plan: Continually evaluate overall results and trends focusing on the “big picture,” and revisit all the above activities in an iterative fashion to improve their application.

23

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved CASE Methodology

24

© 2008 The MITRE Corporation. All rights reserved Putting It All Together… See Notes Page 1. Characterize 2. Characterize Your Your Environment Current Approach

4. Characterize Your New Approach 3. Apply CASE Methodology

25

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved Outline

 Big Ideas  Basics of ESE – Making it an engineering discipline  Next generation systems thinking  An example

26

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved Example of a Future Way of Achieving ESE Mission Capability in Systems Acquisition

 Determine the fundamental unique value (FUV) of each system  Share FUVs in a developers’ network and concentrate on interfaces to make your FUV as widely available as possible  Users will be able compose urgent operational capabilities quickly through selecting FUVs to do the job  A working analogy to have is that of LEGO blocks, where each LEGO block corresponds to a FUV  Developers get rewarded only after their LEGO block is utilized in this way in the field – This is a principle of CASE: Reward Results – Note this is not the way the acquisition world works today! – Other CASE activities apply, as well

27

[White, 2008b] and [Norman-White, 2008] © 2008 The MITRE Corporation. All rights reserved Takeaways  Complex systems include people, and we must bring more humility to engineering the environment of enterprises.

 A complementary, non-traditional mindset is critical to furthering progress in systems engineering.

 Exchanging definitions and terminology is critical first step.

 Use multiple perspectives of each system, SoS, and enterprise to help achieve better understandings.

 In complex enterprises, balance opportunities and risks.

 Endeavor to shape the environment of the system, SoS, or enterprise.

 Continually characterize your environment and what you’re doing about it.

28

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved List of References

[Axelrod-Cohen, 2001] Axelrod, R., and M. D. Cohen, July 2001, Harnessing Complexity – Organizational Implications of a Scientific Frontier, Basic Books

[Gharajedaghi, 1999] Gharajedaghi, Jamshid, 1999, Systems Thinking – Managing Chaos and Complexity: A Platform for Designing Business Architecture, Butterworth Heinemann, Boston

[Hillson, 2004] Hillson, David, 2004, Effective Opportunity Management for Projects, Risk Doctor & Partners, Petersfield, Hampshire, United Kingdom, Marcel Dekker, Inc., New York

[Knowles, 2002] Knowles, Richard N., 2002, The Leadership Dance—Pathways to Extraordinary Organizational Effectiveness, 3rd Edition, The Center for Self-Organizing Leadership, Niagara Falls, NY

[Kuras-White, 2005] Kuras, M. L., and B. E. White, 11 July 2005, “Engineering Enterprises Using Complex-System Engineering,” INCOSE 2005 Symposium, 10-15 July 2005, Rochester, NY

[Norman-White, 2008] Norman, D. O., and B. E. White, “‘So…,’ asks the Chief Engineer ‘What do I go do?’,” 2nd Annual, IEEE Systems Conference, Montreal, Quebec, Canada, 7-10 April 2008

[Sheard, 2008] Sheard, Sarah A. (Principal, 3rd Millennium Systems, LLC), 13 February 2008, “Principles of Complex Systems for Systems Engineering,” Talk, INCOSE New England Chapter Meeting

[White, 2008b] White, B. E., 7 March 2008, “Fundamentals of Enterprise Systems Engineering (ESE),” Talk for Canadian Aerospace Systems Course Visit to MITRE/Bedford, MA

[White, 2008a] Systems Engineering Lexicon, February 2008, (hosted by PHP Wiki ixwebhosting.com via Taylor & Francis Publisher): http://enterprise-systems-engineering.com/phpwiki/

[White, 2008] White, Brian (Theme Editor), January 2008, “Systems Science: Deepening Our Understanding of the Theory and Practice of Systems Engineering,” INCOSE INSIGHT, Vol. 11, Issue 1

29 [White, 2008b] (update of similar 6 March 2006 briefing) © 2008 The MITRE Corporation. All rights reserved List of References (Concluded)

[White, 2007b] White, B. E., 9 November 2007, “Let’s Give More Emphasis to Opportunities in ‘Complex’ ‘Enterprise’ Environments,” Conference on Advanced Risk Management and Its Applications to Systems Engineering, Hosted by the Hampton Roads Area Chapter of International Council on Systems Engineering (HRA INCOSE), Newport News, VA, 8-9 November 2007

[White, 2007a] White, B. E., 18 May 2007, “Let’s Talk More About Opportunities in Uncertainty Management!,” Project Risk Symposium, San Francisco, CA, 16-18 May 2007

[White, 2007] White, B. E., 11 April 2007, “On Interpreting Scale (or View) and Emergence in Complex Systems Engineering,” 1st Annual IEEE Systems Conference, Honolulu, HI, 9-12 April 2007

[White, 2006a] White, B. E., 26 October 2006, “Fostering Intra-Organizational Communication of Enterprise Systems Engineering Practices,” National Defense Industrial Association 9th Annual Systems Engineering Conference, San Diego, CA, October 23-26, 2006

[White, 2006] White, B. E., 11 July 2006, “Enterprise Opportunity and Risk,” INCOSE 2006 Symposium, 9-13 July 2006, Orlando, FL

[White, 2005] White, B. E., 26 October, “A Complementary Approach to Enterprise Systems Engineering,” National Defense Industrial Association, 8th Annual Systems Engineering Conference, San Diego, CA, 24-27 October 2005

30

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved Backup

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved Some Definition Dependencies

32 [White, 2008b], [White, 2006a] and [White, 2008a] © 2008 The MITRE Corporation. All rights reserved See Notes Page Complexity Terms: View, Complexity, Emergence

 View: A human conceptualization consisting of scope, granularity, mindset, and timeframe  Complexity: Description of the ultimate richness of an entity that – Continuously evolves dynamically through self-organization of internal relationships – Requires multi-view analysis to perceive different non-repeating patterns of its behavior – Defies methods of pre-specification, prediction, and control

 Note: Complexity as really a continuum extending from its lowest degree, complication, say, to its higher degree, intended here.  Emergence: Something unexpected in the collective behavior of an entity within its environment, not attributable to any subset of its parts, that is present (and observed) in a given view and not present (or observed) in any other view. – Notes: Some people employ a broader definition where things that emerge can be expected as well as unexpected. Emergence can have benefits or consequences.

33 [White, 2008b], [White, 2006a], [White, 2007], and White, 2008a] © 2008 The MITRE Corporation. All rights reserved See Notes Page System Terms: System and SoS

 System: An interacting mix of elements forming an intended whole greater than the sum of its parts. – Features: These elements may include people, cultures, organizations, policies, services, techniques, technologies, information/data, facilities, products, procedures, processes, and other human-made (or natural) entities. The whole is sufficiently cohesive to have an identity distinct from its environment.  System of Systems (SoS): A collection of systems that functions to achieve a purpose not generally achievable by the individual systems acting independently. – Features: Each system can operate independently (in the same environment as the SoS) and is managed primarily to accomplish its own separate purpose.

34 [White, 2008b], [White, 2006a] and [White, 2008a] © 2008 The MITRE Corporation. All rights reserved System Terms (Concluded): Complex See Notes Page System, CAS, and Enterprise

 Complex System: An open system with continually cooperating and competing elements. – Features: Continually evolves and changes according to its own condition and external environment. Relationships among its elements are difficult to describe, understand, predict, manage, control, design, and/or change.

 Notes: Here “open” means free, unobstructed by artificial means, and with unlimited participation by autonomous agents and interactions with the system’s environment.  Complex Adaptive System (CAS): Identical to a complex system.  Enterprise: A complex system in a shared human endeavor that can exhibit relatively stable equilibria or behaviors (homeostasis) among many interdependent component systems. – Feature: An enterprise may be embedded in a more inclusive complex system.

35 [White, 2008b], [White, 2006a] and [White, 2008a] © 2008 The MITRE Corporation. All rights reserved See Notes Page Engineering Terms: Engineering, Enterprise Engineering, and Systems Engineering

 Engineering: Methodically conceiving and implementing viable solutions to existing problems.  Enterprise Engineering: Application of engineering efforts to an enterprise with emphasis on enhancing capabilities of the whole while attempting to better understand the relationships and interactive effects among the components of the enterprise and with its environment.  Systems Engineering: An iterative and interdisciplinary management and development process that defines and transforms requirements into an operational system. – Features: Typically, this process involves environmental, economic, political, social, and other non-technological aspects. Activities include conceiving, researching, architecting, utilizing, designing, developing, fabricating, producing, integrating, testing, deploying, operating, sustaining, and retiring system elements.

36 [White, 2008b], [White, 2006a] and [White, 2008a] © 2008 The MITRE Corporation. All rights reserved Engineering Terms (Concluded): TSE, ESE, See Notes Page and Complex Systems Engineering

 Traditional Systems Engineering (TSE): Systems engineering but with limited attention to the non-technological and/or complex system aspects of the system. – Feature: In TSE there is emphasis on the process of selecting and synthesizing the application of the appropriate scientific and technical knowledge in order to translate system requirements into a system design.  Enterprise Systems Engineering (ESE): A regimen for engineering “successful” enterprises. – Feature: Rather than focusing on parts of the enterprise, the enterprise systems engineer concentrates on the enterprise as a whole and how its design, as applied, interacts with its environment.  Complex Systems Engineering (CSE): ESE that includes additional conscious attempts to further open an enterprise to create a less stable equilibrium among its interdependent component systems. – Feature: The deliberate and accelerated management of the natural processes that shape the development of complex systems.

37 [White, 2008b], [White, 2006a] and [White, 2008a] © 2008 The MITRE Corporation. All rights reserved Different Patterns At Different Scales

They are all the same things … or are they?

38

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved Different Patterns At Different Scales (Continued)

Data Information Knowledge Wisdom

The question is much harder with people.

39

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved Different Patterns At Different Scales (Concluded)

Scales Groupings Patterns Descriptions

Broad Descriptions Enterprise Net-centric Enterprise Sub-enterprise Capabilities-based Systems Engineering Evolutionary Development

Capabilities Constraints

Tens of Mission Descriptions System of Systems System of Systems Mission Capabilities-based Engineering (Mission Strings) … Composition of Systems

Functionality Constraints … Hundreds … Functional Descriptions System of Independent Detailed Specifications Engineering Systems … Detailed Requirements

40

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved See Notes Page Risk/Opportunity Representation on Probability/Impact Grid

High “Attention Arrow” High

Probability Probability

Po Qo Medium Medium

Risks Opportunities Low Low

Low Medium High High Medium Low Negative Impact Positive Impact Consequence of Failure Benefit of Success B Cf s

41 [White, 2008b], [White, 2007b] and [White, 2007a] (after [Hillson, 2004], p. 126) © 2008 The MITRE Corporation. All rights reserved Some Questions for the Nine Enneagram Perspectives*

 Point 0 (Identity): Who are we? What is our Identity? What is our history, individually and collectively?  Point 1 (Intention): What are we trying to do? What are our Intentions? What is the future potential?  Point 2 (Issues): What are the problems and Issues facing us? What are our dilemmas, paradoxes and questions?  Point 3 (Relationships): What are our Relationships like? How are we connected to others we need in the system? What is the quality of these connections? Are there too many or too few of them?  Point 4 (Principles and Standards): What are our Principles and Standards of behavior? What are our ground-rules, really? What are the un-discussable behaviors that go on, over and over?  Point 5 (Work): What is our Work? On what are we physically working?

______42 [White, 2008b] * [Knowles, 2002, pp. 28-29] Note: Above text has been changed to read as “us,”© 2008not The “them.” MITRE Corporation. All rights reserved Some Questions for the Nine Enneagram Perspectives* (Concluded)

 Point 6 (Information): Do we know what's going on? How do we create and handle Information?  Point 7 (Learning): Are we Learning anything? What are our Learning processes? What is the future potential?  Point 8 (Structure and Context): How are we organized? What is our Structure? Where does the energy come from that makes things happen in our organization? Is our hierarchy deep or flat? What's happening in the larger environment, in which we're living and trying to thrive? Who are our competitors and what are they doing? What is the Context or surrounding environment in which we are living and working?  Point 9 (Our New Identity): After we’ve moved through these questions, how has our Identity changed? Have we expanded and grown? What new things do we now know? What new skills do we now have?

______43 [White, 2008b] * [Knowles, 2002, pp. 28-29] Note: Above text has been changed to read as “us,”© 2008not The “them.” MITRE Corporation. All rights reserved Set Theory View of Engineering Disciplines See Notes Page “SE – CSE”= The portion of TSE that should NOT be used in CSE Engineering SE CSE ESE

(An enterprise is a complex system but the converse is not necessarily true.)

44

[White, 2008b] © 2008 The MITRE Corporation. All rights reserved