Methodology Tailoring for Development Time Optimization Anthony Millán JHU/APL 3/20/2019 Methodology Tailoring for Development Time Optimization 1 Agenda • Background • Definitions • System Complexity, DARPA, and the META Design Flow • Potential to Improve SE Practice • Remaining Problems • Tailoring a SE Methodology for the META Design Flow • Research Approach • META Design Flow Project SDLC • Selecting and Tailoring a Methodology • Tailored Methodology • Methodology Validation • Research Conclusions • Additional Efforts • 2018 INCOSE Systems Engineering in Healthcare • Follow-up Research 3/20/2019 Methodology Tailoring for Development Time Optimization 2 Definitions • Model-Based Systems Engineering (MBSE) – Systems Engineering approach that focuses on the use of descriptive models to support the definition and capture of system design decisions • System Development Lifecycle (SDLC) – Stepwise evolution of a new system from concept through development and on to production, operation and ultimate disposal • Design Flow – Portion of the SDLC that encompasses development of the conceptual and detailed design • Methodology – Overarching set of Systems Engineering processes iteratively applied throughout the SDLC stages • Process – Set of organized activities that define Systems Engineering effort focus and transforms inputs into outputs • Activity – Detailed Systems Engineering task performed as part of a process and adjusted for a specific SDLC stage 3/20/2019 Methodology Tailoring for Development Time Optimization 3 The Challenge: System Complexity • The value of Systems Engineering (SE) – historically managing complexity and change • “As complexity and change continue to escalate in products, services, and society; reducing the risk associated with new systems or modifications to complex systems continues to be a primary goal of the systems engineer”1 However… • The systems community continues to face developing challenges regarding complexity • “Mission complexity is growing faster than our ability to manage it… increasing mission risk from inadequate specifications and incomplete verification”2 • “System design emerges from pieces, rather than from architecture… resulting in systems that are brittle, difficult to test, and complex and expensive to operate”2 1INCOSE SE Handbook, INCOSE, 2015 2INCOSE SE Vision 2025, INCOSE, 2014 3/20/2019 Methodology Tailoring for Development Time Optimization 4 The Focus: Military System Complexity Military system complexity challenges identified by DARPA: • Unexpected interactions • SE has not (fundamentally) changed in 50 years • Unlike other industries, development time of military systems increases proportionally with complexity Source: Paul Eremenko, Formal Model-Based Design & Manufacture: A Template for Managing Complexity in Large-Scale Cyber-Physical Systems, 2013 3/20/2019 Methodology Tailoring for Development Time Optimization 5 The Source: Systems Engineering DARPA attributed some of these problems to shortcomings in the application of SE to military system development • Functional decomposition introduces unnecessary boundaries (stovepipes) • Inconsistent optimization goals (cost, SWaP) • Disincentivized design abstraction • Integration results in “brittle” point design architectures • Unknown emergent behaviors • Traditional V&V methods unable to scale with highly complex systems because of the large number of possible system states Source: Paul Eremenko, Formal Model-Based Design & Manufacture: A Template for Managing Complexity in Large-Scale Cyber-Physical Systems, 2013 3/20/2019 Methodology Tailoring for Development Time Optimization 6 DARPA’s Solution: META Design Flow As a response, DARPA started the Adaptive Vehicle Make (AVM) Portfolio • Programs: META, iFab, VehicleFORGE • Purpose: Transform design, analysis, verification and manufacture of complex systems to compress the development timelines by at least 5X META design flow – New system development method proposed by DARPA as part of the META program • Prerequisites: 1. Formalized design language 2. Defined system-level requirements and concept 3. Detailed component model library Source: DARPA - iFAB Foundry / FANG Proposers’ Day 3/20/2019 Methodology Tailoring for Development Time Optimization 7 DARPA’s Solution: META Design Flow META design flow stages: • Develop an optimized system-level logical architecture taking into account the requirements and the available components • Define physical architecture alternatives from all possible component combinations • Execute the architecture alternatives through automated analysis tools to exclude from consideration poor performers • Increase modeling and analysis fidelity and iterate until final architecture is selected Source: ISIS/Vanderbilt University - The META Toolchain: Accomplishments and Open Challenges 3/20/2019 Methodology Tailoring for Development Time Optimization 8 Potential to Improve SE Practice Common MBSE Benefits Additional META Design Flow Benefits • Promotes use of modular, open systems approach • Composing system design from pre-defined lower (MOSA) level models • Promotes design of tightly bound, loosely coupled • Reduced development time for complex systems architectures • Correct-by-construction designs • Shift in SE effort to early stages of development • Optimizing for system complexity and adaptability • Increased level of abstraction in design instead of functional performance metrics • Model-based verification • Increasing return on investment for repeated uses • Faster requirement-based trade studies The META design flow leverages the benefits of MBSE methods and expands beyond them 3/20/2019 Methodology Tailoring for Development Time Optimization 9 Example: Improving Trade Studies Traditional Trade Study Process Trade Study Using META Design Flow Entire Component Design Space Entire Component Design Space Candidate Candidate Architectures Architectures (optimum) 1 (optimum) 1 3 Available Architectures 2 2 Selected Selected Architecture Architecture 1. Component model library continuously updated and expanded 1. Candidate architectures are selected for consideration from SME & SE judgment 2. Unfeasible architectures excluded through requirements-based analysis 2. Final architecture is selected from an analysis of the candidates 3. Final architecture selected through higher fidelity analysis 3/20/2019 Methodology Tailoring for Development Time Optimization 10 Example: Increased RoI Initial System Development Effort Subsequent Efforts within Domain Entire Component Design Space Entire Component Design Space Candidate Candidate Architectures Architectures 1 3 Available Available Architectures Architectures 2 1 2 Selected Selected Architecture Architecture 1. Component model library continuously updated and expanded 1. Larger component model database results in more 2. Unfeasible architectures excluded through requirements-based available architectures analysis 2. Increased model fidelity accelerates design optimization 3. Final architecture selected through higher fidelity analysis 3/20/2019 Methodology Tailoring for Development Time Optimization 11 Remaining Problems “If application of the META Design Flow has so many potential benefits, why has it not been broadly applied in fields outside of military system development?” • Limited exposure outside of AVM programs • Constraining prerequisites • Limited theoretical foundation Research Problem Summary: • The META design flow proposes a transformation of Systems Engineering design methods to reduce the development time of complex and adaptable systems • This approach could improve system development processes in different industries • Several barriers have limited the META design flow’s adoption in other domains • Literature review yielded that additional research on the integration of the META design flow into the broader scope of Systems Engineering methodologies could provide a framework for its incorporation into other domains 3/20/2019 Methodology Tailoring for Development Time Optimization 12 Research Approach Proposed Research: • Incorporate the META design flow into a tailored end-to-end SE methodology to expand its applicability to other domains Approach: 1. Study SDLC of a project using the META design flow 2. Study existing methodologies to select one for tailoring 3. Tailor the selected methodology following guidance from the Tailoring Process described in chapter 8 of the INCOSE SE Handbook: • Derive modifications from SDLC analysis and unique domain needs • Modify activities within the original processes • Add new processes, remove processes, and combine processes 3/20/2019 Methodology Tailoring for Development Time Optimization 13 META Design Flow Project SDLC • By incorporating the META design flow into the SDLC, parts of Concept Definition, Advanced Development and Engineering design stages are combined and all component-level development is removed from SE consideration 3/20/2019 Methodology Tailoring for Development Time Optimization 14 Review of Available SE Methodologies 14 SE and MBSE methodologies were evaluated: Evaluation Criteria: • MIL-STD-449B • Research goal compatibility • IEEE-1220 • EIA/ANSI 632 • META design flow compatibility • ISO/IEC/IEEE 15288 (INCOSE) • Process Structure • Kossiakoff et al. SE Method Traditional SE • Iteration approach • Top-Down Systems Engineering • Recursion approach • Waterfall Software Engineering • Spiral Software Engineering • SDLC relationship • Harmony-SE • Coverage of SDLC stages • INCOSE OOSEM • SDLC dependence/independence