AGILE ASSESSMENT GUIDE Best Practices for Agile Adoption and Implementation From September 28, 2020 to September 27, GAO-20-590G 2021, GAO is seeking input and feedback on this Exposure Draft from all interested September 2020 parties. See page 3 for more information. Contents Preface 1 Introduction 4 Chapter 1 Background 7 Chapter 2 Agile Adoption Challenges in the Federal Government and Actions Taken in Response 14 Chapter 3 Agile Adoption Best Practices 26 Team Dynamics and Activities 30 Best practice: Team composition supports Agile methods 30 Best practice: Work is prioritized to maximize value for the customer 35 Best practice: Repeatable processes are in place 40 Program Operations 44 Best practice: Staff are appropriately trained in Agile methods 44 Best practice: Technical environment enables Agile development 46 Best practice: Program controls are compatible with Agile 49 Organization Environment 51 Best practice: Organization activities support Agile methods 52 Best practice: Organization culture supports Agile methods 55 Best practice: Organization acquisition policies and procedures support Agile methods 61 Best Practices Checklist: Adoption of Agile Methods 63 Chapter 4 Overview of Agile Execution and Controls 65 Chapter 5 Requirements Development and Management in Agile 75 Elicit and Prioritize Requirements 80 Refine and Discover Requirements 83 Ensure Requirements are Sufficiently Complete, Feasible, and Verifiable for the Current State of the Program 84 Balance Customer Needs and Constraints 86 Test and Validate the System as it is Being Developed 89 Manage and Refine Requirements 90 Page i GAO-20-590G GAO Agile Assessment Guide Maintain Traceability in Requirements Decomposition 91 Ensure Work is Contributing to the Completion of Requirements 92 Best Practices Checklist: Requirements Development 93 Chapter 6 Agile and the Federal Contracting Process 95 Tailor contract structure and inputs to align with Agile practices 98 Incorporate Agile metrics, tools, and lessons learned from retrospectives during the contract oversight process 104 Integrate the program office and the developers 110 Best Practices Checklist: Contracting for an Agile Program 115 Chapter 7 Agile and Program Monitoring and Control 116 Work breakdown structure in an Agile environment 117 Cost estimating best practices in an Agile environment 119 Agile measures and documenting the cost estimate 120 Considerations for developing a cost estimate for an Agile program 125 Scheduling best practices in an Agile environment 130 Agile measures and scheduling best practices 130 Considerations for scheduling an Agile program 133 Earned value management best practices in an Agile environment 137 Agile measures and Earned Value Management 139 Considerations for applying earned value management to an Agile program 144 Best Practices Checklist: Agile and Program Monitoring and Control 149 Chapter 8 Agile Metrics 150 Identify key metrics based on the program’s Agile framework 152 Ensure metrics align with organization-wide goals and objectives 155 Establish and validate metrics early and align with incentives 156 Establish management commitment 158 Commit to data-driven decision making 160 Communicate performance information frequently and efficiently 163 Best Practices Checklist: Agile Metrics 165 Appendix I Scope and Methodology 167 Page ii GAO-20-590G GAO Agile Assessment Guide Appendix II Key Terms 169 Appendix III Related Terms 180 Appendix IV Auditor’s Key Questions and Effects 181 Appendix V Common Agile Frameworks 218 Appendix VI Debunking Agile Myths 232 Appendix VII Background for Case Studies and Agile in Actions 236 Appendix VIII Specialists Who Helped Develop this Guide 244 Appendix IX GAO Contacts and Staff Acknowledgments 246 References 247 Tables Table 1: Description of Commonly-Used Agile Frameworks 11 Table 2: Iterative Software Challenges, as Reported by Federal Agencies 14 Table 3: Laws, Policy, Guidance, Reports, and Entities Established to Address Agile Challenges 20 Table 4: Recent GAO Reports Highlighting Agile Challenges 22 Table 5: Summary of Agile Adoption Best Practices 29 Table 6: Manual Coding Quality Assurance Methods 42 Page iii GAO-20-590G GAO Agile Assessment Guide Table 7: Summary of Agile Requirements Management Best Practices 80 Table 8: Summary of Agile and Contracting Best Practices 98 Table 9: Differences between Statement of Objectives and Statement of Work 100 Table 10: Examples of Agile Metrics by Metric Category 105 Table 11: 12-Step Cost Estimating Process and Agile Cadence Examples 121 Table 12: Characteristics of a Reliable Cost Estimate and Agile Measures 124 Table 13: Comparison of Consistent Sizing and Relative Sizing 126 Table 14: 10-Step Schedule Estimating Best Practices and Agile Cadence Examples 131 Table 15: GAO Earned Value Management Best Practices 138 Table 16: 13 Earned Value Management Activities and Agile Examples 139 Table 17: Summary of Agile Metrics Best Practices 151 Table 18: Terms Used In This Guide and Related Terms 180 Table 19: DevOps Principles 219 Table 20: Disciplined Agile Roles and Responsibilities 220 Table 21: Disciplined Agile Principles 221 Table 22: Dynamic Systems Development Method Principles 222 Table 23: eXtreme Programming Activities 223 Table 24: eXtreme Programming Values 224 Table 25: Lean Software Development Principles 226 Table 26: Scaled Agile Framework Roles and Responsibilities 228 Table 27: Scaled Agile Framework Principles 228 Table 28: Scrum Team Structure 230 Table 29: Scrum Principles 231 Table 30: Case Studies Drawn from GAO Reports Used In this Guide 236 Table 31: Agile in Action Drawn from GAO Interviews 243 Figures Figure 1: Comparison of Agile and Waterfall Methods for Developing Software 8 Figure 2: Overview of Agile Adoption Best Practices 28 Figure 3: Relationship between the Agile Team and Customers 32 Figure 4: Agile Planning Levels 67 Figure 5: Comparison of Traditional and Agile Development Program Management Constraints 70 Page iv GAO-20-590G GAO Agile Assessment Guide Figure 6: Overview of Requirements Management Best Practices 79 Figure 7: Backlog Decomposition for an Agile Program 88 Figure 8: Overview of Agile and Contracting Best Practices 97 Figure 9: Comparison of Waterfall and Agile Programs’ Review Cycles 109 Figure 10: Roles When Planning, Managing, and Executing an Agile Contract 111 Figure 11: Work Breakdown Structure in an Agile Program 118 Figure 12: Traditional and Agile Earned Value Management Tracking Methods 143 Figure 13: Comparison of Traditional and Agile EVM Products 145 Figure 14: Example of Measuring Earned Value for an Agile Feature 146 Figure 15: Overview of Agile Metrics Best Practices 151 Figure 16: Example of a Cumulative Flow Diagram 155 Figure 17: Timeline of Agile Development 218 Figure 18: Kanban Board 225 Abbreviations ALIS Automated Logistics Information System COR contracting officer’s representative CPIC Capital Planning and Investment Control DHS Department of Homeland Security DevOps Development and Operations DAD Disciplined Agile Delivery DOD Department of Defense DSDM Dynamic Systems Development Method EVM earned value management ELIS Electronic Immigration System program FAR Federal Acquisition Regulation FDD Feature Driven Development FEMA Federal Emergency Management Agency FITARA Federal Information Technology Acquisition Reform Act FOC full operational capability G2 Generation 2 (NNSA Program Management Information Systems) GSA General Services Administration GMM Grants Management Modernization Page v GAO-20-590G GAO Agile Assessment Guide Abbreviations Continued ICE U.S. Immigration and Customs Enforcement INVEST Independent, Negotiable, Valuable, Estimable, Small, Testable LeSS Large Scale Scrum MVP Minimum viable product MUOS Mobile User Objective System program MoSCoW Must have, should have, could have, won’t have (sometimes would have) NDAA National Defense Authorization Act NNSA National Nuclear Security Administration OMB Office of Management and Budget OTC Office of Transformation Coordination RFP request for proposal SAFe Scaled Agile Framework SEVIS Student and Exchange Visitor Information System program SOO Statement of Objectives SOW Statement of Work Space C2 Air Force Space Command and Control program TIM Technology Infrastructure Modernization TSA Transportation Security Administration USCIS U.S. Citizenship and Immigration Services USDS U.S. Digital Services WBS work breakdown structure XP eXtreme Programming This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page vi GAO-20-590G GAO Agile Assessment Guide Letter 441 G St. N.W. Washington, DC 20548 The U.S. Government Accountability Office (GAO) is responsible for, Preface among other things, assisting Congress in its oversight of the executive branch, including assessing federal agencies’ management of information technology (IT) systems. In prior audits, GAO has reported that federal agencies faced challenges in developing, implementing, and maintaining their IT investments. All too frequently, agency IT programs have incurred cost overruns and schedule slippages while contributing little to mission- related outcomes. Accordingly, in February 2015, GAO added the government’s management of