Systems Engineering
Total Page:16
File Type:pdf, Size:1020Kb
What T&E’rs Need to Know About A Webinar Based On A Short Course By: Dr. W. David Bell Dr. C. David Brown SYSTEMS ENGINEERING A n d PROGRAM MANAGEMENT A n d W h y This Webinar Highlights Selected Topics from Our 3 Day Short Course • Overview of Program/Project Management (PM) • Overview of Systems Engineering (SE) • Test and Evaluation (T&E) and Interaction with PM and SE • Modeling and Simulation & T&E • Current Trends in T&E – T&E early in programs – Verify vs Validate = DT vs OT – Integrated T&E – T&E of immature technologies – Agile acquisition T&E implications • Case Studies - Army Future Combat Systems & Navy Littoral Surveillance Radar System • DOD Acquisition – Process, Life Cycle, Regulations, Directives, Guidance, & Instructions • Special Topics – Software SE and T&E – Human Systems Integration – Unmanned and Autonomous Sys SE and T&E – Cyber and Info Assurance SE and T&E 31 May 2013, 2100 2 Dave Bell Principal Multi-Discipline Systems Engineer MITRE Corp [email protected] Things done: • Data Analysis • Flight test direction - DT and OT • Scientific experiments • Systems engineering from beginning to end – Requirements definition – Tech development, tech maturing – I&T at all levels - assembly, subsystem & system • Managed S&T programs • VP of operations • Adjunct Professor /Lecturer of systems engineering – SMU, JHU Education • BS Physics, MBA, D. Engineering 31 May 2013, 2100 3 Dave Brown Consulting Systems Engineer MITRE Corp and Institute for Defense Analyses [email protected] Things done: • Colonel, US Army Signal Corps • Developing DT instrumentation and test facilities • Development of M&S for test support – Army Virtual Proving Ground • Live Fire T&E • Army Senior Executive – Director of Test and Technology, Army Developmental Test Command • Deputy PM/Executive Director for Test, Army Future Combat Systems Program • Instructor – JHU Masters in Systems Engineering Education • BS, MS, PhD Electrical Engineering • Masters in National Security Policy, ICAF 31 May 2013, 2100 4 C Bottom Line Up Front What PM elements do T&E’rs need to understand? – Leadership, authority – Stakeholders – Risk management – Planning, monitoring, and control – Work breakdown structure (WBS) – Budgeting – Scheduling – Earned Value Management (EVM) Why: – All of the above involve some information for which T&E is the source – Accomplishing T&E involves work (WBS), resources (Budget), and time (Schedule) – Information from T&E is critical to stakeholder communication, risk management, monitoring, and assessing work completion for EVM – T&E planning, resourcing, execution, and completion must be “program managed” 31 May 2013, 2100 5 Bottom Line Up Front C What SE elements do T&E’rs need to understand? – Needs and definition – Requirements analysis – Synthesis – Integration – Maturity/Readiness – Trades – Verification – Validation – Risk management Why: – All of the above involve some information for which T&E is the source – The test system must be system engineered – Early T&E involvement in programs involves T&E and SE integration – Pushed by GAO and proven by successful programs 31 May 2013, 2100 6 Bottom Line Up Front C And most important: – T&E’rs must have a thorough knowledge and understanding of the total system and system management process to understand the information and the information needs and perspectives of each stakeholder, So that: – T&E can provide each stakeholder with information that is: •Required •Relevant •Complete •Accurate •Appropriate •Understandable •In the Correct Context 31 May 2013, 2100 7 C Our Challenging World • Terrorism • Global culture clashes • Information overload • Disease – Health Care • Infrastructure • Technical Complexity • Business and Finance Problem Attributes • Technical • Size - Scope • Complexity • Importance • Social, political, fiscal Systems Engineering and Management is increasingly the field expected to solve the problems. 31 May 2013, 2100 8 C A Systems Approach Operational Data Collection Lessons Learned Test & Evaluation Capabilities Improvement Product Development & Needs Definition Production Data Collection Mission Performance Analysis Operations Technology Knowledge Prototype Development Performance Demonstration Critical Field Experiments Enabling Science & Technology Hypothesis, Concept Development Trade-offs & Critical Experiments Technology Knowledge Transfer Modeling9 and Simulations This slide adapted from a brief by: 31 May 2013, 2100 Dr. Samuel Seymour, JHU Applied Physics Lab 9 C System Management Systems Program/Project Engineering Management Configuration Management System Design & Engineering Quality Control Project Administration Interface Contract & Support Management Management Analysis & Production Data Management Evaluation Management Integration Technical Performance, & Test Cost, Schedule Test & Evaluation 31 May 2013, 2100 10 Senator John McCain, 15 December 2011 To be clear, the military-industrial-congressional complex does not cause programs to fail. But, it does help create poorly-conceived programs — programs that are so fundamentally unsound that they are doomed to be poorly executed. And, it does help keep them alive—long after they should have been ended or restructured. By “poorly conceived”, I mean major programs that are allowed to begin despite having insufficiently defined requirements; unrealistic cost or schedule estimates; immature technology or too much manufacturing and integration risk; or unrealistic performance expectations. By “poorly executed”, I am referring to programs that poorly perform because of, among other things, unanticipated design, engineering, manufacturing or technology problems. These sorts of programs should either never have been started to begin with or should have been significantly restructured or terminated at the end of the day. Specifically, the military-industrial-congressional complex helps ensure that poorly- conceived programs get on rails—and stay there—with “production money” when they are supposed to still be in development. And, for Industry and many of their sponsors in the Pentagon and on the Hill, that’s desirable because it is far more difficult to restructure or terminate a production program—even one that’s performing poorly—than one that’s in development. Honorable Frank Kendall (USD(AT&L)), ITEA Journal, March 2013 The purpose of developmental testing is simple; to provide data to program leadership so that good decisions can be made as early as possible. I have a sign outside my office that is a quote from G. Edwards Deming: “In God we trust, all others must bring data.” It is our developmental testers who “bring the data” that is needed to make sound decisions during product development. Programs are organized in various ways, but whatever the specific organizational model, testing is the source of the crucial information that provides feedback to program management, chief engineers, lead system engineers, integrated product teams, and military users on whether their designs meet requirements or not. Five precepts – DT&E should: Contribute to program efficiency and effective execution, Provide relevant information as early as possible, Integrate DT&E planning across the product life cycle, Focus on support to internal program decisions and verification of compliance with requirements, and Use DT&E to improve the efficiency and validity of OT&E. We sometimes fail to conduct adequate DT&E prior to the decision to start production. About a year ago, I called a particular decision to enter production on an aircraft program without flight testing “acquisition malpractice.” If a product enters production before the design is stable, the resulting waste in cost increases and schedule slips can be dramatic, and the program is much more likely to be canceled. I stress solid, well defined DT&E results as an important prerequisite for this decision because the pressure to enter production can be overwhelming, and doing so prematurely has major consequences. Working with program and engineering leadership as key members of the management team, developmental testers provide the information that makes program success possible and much more probable. 12 31 May 2013, 2100 C “The Test and Evaluation (T&E) process is an integral part of the Systems Engineering (SE) process, which identifies levels of performance and assists the developer in correcting deficiencies. It is a significant element in the decision-making process, providing data that support trade-off analysis, risk reduction, and requirements refinement.”* Both SE and T&E are integral parts of the System/Program Management Process where SE provides the technical direction and T&E provides information to support the technical elements above as well as management processes and information to stakeholders of the System/Program Management Process. These include PMs, decision makers, and Congress. *DAU Test and Evaluation Management Guide 31 May 2013, 2100 13 Attributes View of a Complex System . Form − Incorporates Many Capabilities − Comprised of Interacting, Diverse Elements − Definable Structure and Interconnections − Bounded; Has Inputs and Outputs . Function − It Does Something Valuable − It is Dynamic—Not Inert . Interfaces − Internal − External This slide adapted from a brief by: 31 May 2013, 2100 Dr. Samuel Seymour, JHU Applied Physics Lab 14 Partitioning View of a Complex System Each system can be broken down into components which in turn can be broken down into smaller components, etc. Air Defense System System Search Radar Subsystem