Engineering Safe and Secure Software Systems For a complete listing of titles in the Artech House Computer Security Series, turn to the back of this book. Engineering Safe and Secure Software Systems C. Warren Axelrod Library of Congress Cataloging-in-Publication Data A catalog record for this book is available from the U.S. Library of Congress. British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library. Cover design by Vicki Kane ISBN 13: 978-1-60807-472-3 © 2013 ARTECH HOUSE 685 Canton Street Norwood, MA 02062 All rights reserved. Printed and bound in the United States of America. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher. All terms mentioned in this book that are known to be trademarks or service marks have been appropriately capitalized. Artech House cannot attest to the accuracy of this information. Use of a term in this book should not be regarded as affecting the validity of any trademark or service mark. 10 9 8 7 6 5 4 3 2 1 To Judy, David, Nicole, Elisabeth, Evan, and Jolie, with wishes for a safer and more secure world for future generations Contents Preface xvii Foreword xxi 1 Introduction 1 Preamble 1 Scope and Structure of the Book 3 Acknowledgments 4 Endnotes 5 2 Engineering Systems 7 Introduction 8 Some Initial Observations 8 Deficient Definitions 11 Rationale 12 What are Systems? 13 Deconstructing Systems Engineering 16 What Is Systems Engineering? 19 vii viii Engineering Safe and Secure Software Systems Contents ix Systems Engineering and the Systems Engineering Management Process 20 The DoD Text 22 Another Observation 22 More on Systems Engineering 23 The Systems Engineering Process (SEP) 23 Summary and Conclusions 26 Endnotes 26 3 Engineering Software Systems 29 Introduction 29 The Great Debate 31 Some Observations 32 Rationale 33 Understanding Software Systems Engineering 34 Deconstructing Software Systems Engineering 34 What Is Software? 35 What Are Software Systems? 36 Are Control Software Systems Different? 42 What is Software Systems Engineering? 42 The Software Systems Engineering Process 44 Steps in the Software Development Process 44 Omissions or Lack of Attention 48 Nonfunctional Requirements 48 Testing Nonfunctional Attributes 49 viii Engineering Safe and Secure Software Systems Contents ix Verification and Validation 49 Creating Requisite Functional and Nonfunctional Data 52 Resiliency and Availability 55 Decommissioning 56 Summary and Conclusions 56 Endnotes 57 4 Engineering Secure and Safe Systems, Part I 59 Introduction 59 The Approach 60 Security Versus Safety 60 Four Approaches to Developing Critical Systems 63 The Dependability Approach 64 The Safety Engineering Approach 65 The Secure Systems Approach 67 The Real-Time Systems Approach 68 Security-Critical and Safety-Critical Systems 68 Summary and Conclusions 70 Endnotes 70 5 Engineering Secure and Safe Systems, Part 2 73 Introduction 73 Approach 75 Reducing the Safety-Security Deficit 76 Game-Changing and Clean-Slate Approaches 77 A Note on Protection 81 Safety-Security Governance Structure and Risk Management 83 x Engineering Safe and Secure Software Systems Contents xi An Illustration 83 The General Development Life Cycle 84 Structure of the Software Systems Development Life Cycle 86 Life Cycle Processes 89 Governance Structure for Systems Engineering Projects 92 Risks of Security-Oriented Versus Safety-Oriented Software Systems 94 Expertise Needed at Various Stages 95 Summary and Conclusions 95 Endnotes 96 6 Software Systems Security and Safety Risk 99 Introduction 99 Understanding Risk 100 Risks of Determining Risk 100 Software-Related Risks 101 Motivations for Risk Mitigation 103 Defining Risk 104 Assessing and Calculating Risk 105 Threats Versus Exploits 107 Threat Risk Modeling 111 Threats from Safety-Critical Systems 114 Creating Exploits and Suffering Events 116 Vulnerabilities 119 Application Risk Management Considerations 120 Subjective vs. Objective vs. Personal Risk 121 Personalization of Risk 122 x Engineering Safe and Secure Software Systems Contents xi The Fallacies of Data Ownership, Risk Appetite, and Risk Tolerance 122 The Dynamics of Risk 124 A Holistic View of Risk 125 Summary and Conclusions 126 Endnotes 128 7 Software System Security and Safety Metrics 131 Introduction 131 Obtaining Meaningful Data 133 Defining Metrics 133 Differentiating Between Metrics and Measures 135 Software Metrics 138 Measuring and Reporting Metrics 140 Metrics for Meeting Requirements 143 Risk Metrics 146 Consideration of Individual Metrics 146 Security Metrics for Software Systems 150 Safety Metrics for Software Systems 151 Summary and Conclusions 152 Endnotes 153 8 Software System Development Processes 157 Introduction 157 Processes and Their Optimization 158 Processes in Relation to Projects and Products/Services 159 xii Engineering Safe and Secure Software Systems Contents xiii Some Definitions 161 Chronology of Maturity Models 164 Security and Safety in Maturity Models 165 FAA Model 165 The +SAFE V1.2 Extension 167 The +SECURE V1.3 Extension 167 The CMMI® Approach 167 General CMMI® 167 CMMI® for Development 168 Incorporating Safety and Security Processes 169 +SAFE V1.2 Comparisons 169 +SECURE V1.2 Comparisons 172 Summary and Conclusions 173 Endnotes 175 9 Secure SSDLC Projects in Greater Detail 177 Introduction 177 Different Terms, Same or Different Meanings 178 Creating and Using Software Systems 180 Phases and Steps of the SSDLC 182 Summary and Conclusions 191 Endnotes 193 10 Safe SSDLC Projects in Greater Detail 195 Introduction 195 Definitions and Terms 196 Hazard Analysis 198 Software Requirements Hazard Analysis 199 Top-Level Design Hazard Analysis 200 Detailed Design Hazard Analysis 201 Code-Level Software Hazard Analysis 201 xii Engineering Safe and Secure Software Systems Contents xiii Software Safety Testing 201 Software/User Interface Analysis 202 Software Change Hazard Analysis 203 The Safe Software System Development Lifecycle 204 Combined Safety and Security Requirements 207 Summary and Conclusions 208 Endnotes 209 11 The Economics of Software Systems’ Safety and Security 211 Introduction 211 Closing the Gap 212 Technical Debt 214 Application of Technical Debt Concept to Security and Safety 215 System Obsolescence and Replacement 217 The Responsibility for Safety and Security by Individuals and Groups 218 Basic Idea 218 Extending the Model 219 Concept and Requirements Phase 219 Design and Architecture Phase 222 Development 223 Verification 224 Validation 224 Deployment, Operations, Maintenance, and Technical Support 225 Decommissioning and Disposal 226 Overall Impression 226 Methods for Encouraging Optimal Behavior 226 Pricing 227 Chargeback 227 Costs and Risk Mitigation 228 Management Mandate 228 xiv Engineering Safe and Secure Software Systems Contents xv Legislation 229 Regulation 229 Standards and Certifications 229 Going Forward 230 Tampering 231 Tamper Evidence 231 Tamper Resistance 232 Tamperproofing 232 A Brief Note on Patterns 234 Conclusions 236 Endnotes 238 Appendix A: Software Vulnerabilities, Errors, and Attacks 239 Ranking Errors, Vulnerabilities, and Risks 240 The OWASP Top Security Risks 241 The CWE/SANS Most Dangerous Software Errors 244 Top-Ranking Safety Issues 244 Enumeration and Classification 246 WASC Threat Classification 248 Summary and Conclusions 250 Endnotes 250 Appendix B: Comparison of ISO/IEC 12207 and CMMI®-DEV Process Areas 253 Appendix C: Security-Related Tasks in the Secure SSDLC 257 Task Areas for SSDLC Phases 258 Involvement by Teams and Groups for Secure SSDLC Phases 262 xiv Engineering Safe and Secure Software Systems Contents xv A Note on Sources 288 Endnotes 288 Appendix D: Safety-Related Tasks in the Safe SSDLC 289 Task Areas for Safe SSDLC Phases 289 Levels of Involvement 309 A Note on Sources 309 Endnotes 313 About the Author 315 Index 317 Preface The best laid plans o’ Mice an’ Men, Gang oft agley ... —Robert Burns, To a Mouse The initial concept for this book arose some 3 to 4 years ago. However, it was quite different from how the book turned out. I had spent much of the previous decade working on application security, particularly software assur- ance. Several years ago, I had the good fortune of being the technical lead on a software assurance initiative for the banking and finance sector, supported by the Financial Services Technology Consortium. The first phase of the proj- ect provided insights from thought leaders from independent software vendors (ISVs), information security tools and services vendors, industry and profes- sional associations, academia, and a number of leading financial institutions. A collection of state-of-the-art practices was assembled, with the intention of us- ing the “best” approaches for assuring the quality of software through industry- sponsored testing. This work provided a substantial amount of the research that was behind the BITS publication Software Assurance Framework [1]. The ultimate goal of the software assurance initiative was to establish a state-of-the-art testing facility for the financial services industry using methods, tools, and services chosen from the initial research. Unfortunately, the financial meltdown of 2008 interceded. Various mergers and restructurings took place, so that attention was turned to other more pressing matters. The financial services industry