United States Government Accountability Office Report to Congressional Committees April 2014 NASA Assessments of Selected Large-Scale Projects GAO-14-338SP April 2014 NASA Assessments of Selected Large-Scale Projects Highlights of GAO-14-338SP, a report to congressional committees Why GAO Did This Study What GAO Found This is GAO’s annual assessment of The National Aeronautics and Space Administration’s (NASA) total portfolio of NASA’s major projects. This report major projects saw cost and schedule growth that remains low compared to provides a snapshot of how well NASA GAO’s first review of the portfolio. Some projects in this year’s portfolio launched is planning and executing its major within their cost and schedule baselines; however, several others are undergoing acquisitions. In 2013, GAO reported replans, which could temper the portfolio's positive performance. For example, that the performance of NASA's major the Mars Atmosphere and Volatile EvolutioN project launched on time and cost projects had improved since GAO’s about $35 million less than its baseline estimate, but NASA officials are reporting first assessment in 2009, due, in part, that issues with the Ice, Cloud, and Land Elevation Satellite-2 project’s primary to some underperforming projects instrument are driving costs to exceed the original baseline by at least 15 launching and some demonstrating percent, and that the project will miss its committed launch date. progress meeting practices that GAO has reported decrease cost and NASA projects have continued to make progress in maturing technologies prior schedule risk. to the preliminary design review. This year, 63 percent of projects met this In response to an explanatory standard, up from only 29 percent of projects in 2010. For example, in statement of the House Committee on preparation for its upcoming confirmation review, one project has matured all 10 Appropriations accompanying the of its critical technologies, which GAO’s past work has shown is important to Omnibus Appropriations Act, 2009, this decrease the likelihood of cost and schedule growth. NASA’s heightened report assesses (1) the current status awareness of reducing technology risk is further evidenced by new guidance of NASA’s portfolio of major projects, aimed at ensuring continued focus on technical maturity. As NASA continues to (2) NASA's progress in developing and undertake more complex projects it will be important to maintain heightened maturing critical technologies (3) attention to best practices to lessen the risk of technology development and efforts NASA has taken to improve continue positive cost and schedule performance. design stability of its projects, and (4) any challenges to NASA’s NASA projects are maintaining steady performance toward meeting GAO’s best management of the portfolio. GAO practices for design stability, and the agency has also increased its focus on assessed 2013 and 2014 data on design stability. GAO has found over past several years that projects have NASA’s 18 major projects and the consistently reported higher percentages of drawings releasable at the critical Commercial Crew program all with an design review and lower percentages of drawing growth after that time, which estimated life-cycle cost of over $250 indicates that project design stability has increased overall. NASA has taken million, such as data on the projects’ steps to enhance its ability to assess design maturity. For example, NASA cost, schedule, technology maturity, implemented three technical indicators to assess design maturity, and projects in design stability, and contracts; the portfolio are tracking the required indicators. Additionally, experts in the analyzed monthly project status space community have identified other design stability metrics, which can be reports; and interviewed NASA and used in tandem with GAO’s and NASA’s indicators in order to provide a more contractor officials. complete and robust assessment of a project’s design stability. What GAO Recommends NASA faces several challenges that could impact its ability to effectively manage GAO is not making any new its portfolio. A primary challenge in the next few years will be to complete a series recommendations in this report, but of complex and expensive projects within constrained budgets and competing provides further evidence to support priorities. Any cost or schedule growth on NASA’s largest, most complex the importance of continuing to take projects, such as the James Webb Space Telescope, could have a ripple effect action on recommendations GAO has across the portfolio. While NASA has implemented a plan for improving its made in prior reports. NASA generally acquisition management, monitoring NASA’s performance against that plan over agreed with GAO’s findings. time will be important in determining if the agency’s efforts to improve its acquisition management practices have become institutionalized. For example, in 2013, two projects experienced significant issues immediately after being View GAO-14-338SP. For more information, confirmed, indicating that neither project had completed an adequate contact Cristina Chaplain at (202) 512-4841 or [email protected]. assessment of risk which is necessary to ensure that the project’s cost and schedule baseline estimates were realistic. United States Government Accountability Office Contents Letter 1 Background 4 Positive Cost and Schedule Trends Continue, but Project Replans Weaken Overall Performance 9 NASA Has Made Progress in Meeting GAO’s Best Practice of Maturing Project Technology Prior to the Preliminary Design Review 13 NASA Has Increased Focus on Design Stability by Tracking Several Metrics 16 NASA Faces Several Challenges That Could Impact Management of the Portfolio 20 Project Assessments 28 Global Precipitation Measurement Mission 31 Gravity Recovery and Climate Experiment Follow On 33 Ice, Cloud, and Land Elevation Satellite-2 35 Interior Exploration using Seismic Investigations, Geodesy, and Heat Transport 37 James Webb Space Telescope 39 Lunar Atmosphere and Dust Environment Explorer 41 Magnetospheric Multiscale 43 Mars Atmosphere and Volatile EvolutioN 45 Orbiting Carbon Observatory 2 47 Origins-Spectral Interpretation-Resource Identification- Security-Regolith Explorer 49 Orion Multi-Purpose Crew Vehicle 51 Soil Moisture Active and Passive 53 Solar Probe Plus 55 Space Launch System 57 Space Network Ground Segment Sustainment 59 Stratospheric Observatory for Infrared Astronomy 61 Surface Water and Ocean Topography 63 Tracking and Data Relay Satellite Replenishment 65 Commercial Crew Program 67 Agency Comments and Our Evaluation 70 Appendix I Objectives, Scope, and Methodology 73 Page i GAO-14-338SP Assessments of Selected Large-Scale Projects Appendix II Selected Major NASA Projects Reviewed in GAO’s Annual Assessments 81 Appendix III Selected Major NASA Projects Reviewed in GAO’s 2014 Annual Assessment 83 Appendix IV Technology Readiness Levels 88 Appendix V Elements of a Sound Business Case 90 Appendix VI Comments from the National Aeronautics and Space Administration 92 Appendix VII GAO Contact and Staff Acknowledgments 96 Tables Table 1: Development Cost and Schedule Growth of Selected Major NASA Projects Currently in the Implementation Phase 11 Table 2: Selected Major NASA Projects and Programs Reviewed in GAO’s Annual Assessments 81 Figures Figure 1: NASA’s Life Cycle for Flight Systems 5 Figure 2: Selected Major NASA Projects Reviewed in GAO’s 2014 Assessment 8 Figure 3: Average Development Cost and Schedule Growth of Selected Major NASA Projects in Implementation Phase, Both Including and Excluding JWST 10 Figure 4: Percentage of Major Projects Meeting GAO Technology Maturity Criteria at the Preliminary Design Review 14 Page ii GAO-14-338SP Assessments of Selected Large-Scale Projects Figure 5: Average Number of Critical Technologies for Projects in Implementation 15 Figure 6: Average Percentage of Releasable Engineering Drawings at Critical Design Review 17 Figure 7: Average Percentage of Drawing Growth after Critical Design Review for Selected Projects from Fiscal Year 2010 through 2014 18 Figure 8: Fiscal Year 2014 Budget Request for JWST, Orion, SLS, Commercial Crew, and All Other Major NASA Projects, 2014 through 2018 21 Figure 9: Illustration of a Sample Project Assessment 30 Page iii GAO-14-338SP Assessments of Selected Large-Scale Projects Abbreviations AFB Air Force Base AFS Air Force Station ATLAS Advanced Topographic Laser Altimeter System CCP Commercial Crew Program CCDev Commercial Crew Development CCDev 2 Commercial Crew Development Round 2 CCiCap Commercial Crew Integrated Capability CCtCap Commercial Crew Transportation Capabilities phase CDR critical design review CNES Centre National d’Etudes Spatiales (French Government Space Agency) CSA Canadian Space Agency DCI Data Collection Instrument(s) DLR German Aerospace Center EFT-1 The first Exploration Flight Test for the Orion vehicle, scheduled for September 2014 EM-1 Exploratory Mission 1, the first non-crewed launch of the Space Launch System and Orion vehicle, planned for December 2017 EM-2 Exploratory Mission 2, the first crewed launch of the Space Launch System and the Orion vehicle, planned for August 2021 EMTGO ExoMars Trace Gas Orbiter ESA European Space Agency EVM earned value management FAR Federal Acquisition Regulation FPI Fast Plasma Instrument GFZ German Research Center for Geosciences GLAST Gamma-ray Large Area Space Telescope GNC LIDAR guidance, navigation, and control light detection and ranging