United States Government Accountability Office GAO Report to Congressional Committees

March 2011 DEFENSE ACQUISITIONS Assessments of Selected Weapon Programs

a

GAO-11-233SP

March 2011 DEFENSE ACQUISITIONS Accountability • Integrity • Reliability Assessments of Selected Weapon Programs

Highlights of GAO-11-233SP, a report to congressional committees

Why GAO Did This Study What GAO Found This is GAO’s ninth annual Since 2008, DOD’s portfolio of major defense acquisition programs has grown assessment of Department of Defense from 96 to 98 programs, and its investment in those programs has grown to (DOD) weapon system acquisitions, $1.68 trillion. The total acquisition cost of the programs in DOD’s 2010 an area that is on GAO’s high-risk list. portfolio has increased by $135 billion over the past 2 years, of which $70 The report is in response to the billion cannot be attributed to quantity changes. A small number of programs mandate in the joint explanatory are driving most of this cost growth; however, half of DOD’s major defense statement to the DOD Appropriations acquisition programs do not meet cost performance goals agreed to by DOD, Act, 2009. It includes observations on the Office of Management and Budget, and GAO. Further, 80 percent of the performance of DOD’s 2010 programs have experienced an increase in unit costs from initial estimates; portfolio of 98 major defense thereby reducing DOD’s buying power on these programs. acquisition programs; data on selected factors that can affect program outcomes; an assessment of Changes in DOD’s Fiscal Year 2010 Portfolio of Major Defense Acquisition Programs over the the knowledge attained by key Past 2 Years (Fiscal Year 2011 Dollars in Billions) junctures in the acquisition process Estimated for a subset of 40 programs, which Estimated Estimated portfolio cost Percentage portfolio cost portfolio cost growth since growth since were selected because they were in in 2008 in 2010 2008a 2008a development or early production; and Total estimated research observations on the implementation and development costs $407 $428 $15 5% of acquisition reforms. To conduct Total estimated this review, GAO analyzed cost, procurement costs 1,089 1,219 121 11 Total estimated schedule, and quantity data from acquisition cost 1,531 1,680 135 9

DOD’s Selected Acquisition Reports Source: GAO analysis of DOD data. and collected data from program aThese columns do not include $6 billion in research and development and $9 billion in procurement offices on performance requirements cost changes for the Ballistic Missile Defense System. DOD does not consider these adjustments to and software development; represent cost growth because the program has been allowed to add 2 years of new funding with technology, design, and each biennial budget. manufacturing knowledge; and the implementation of DOD’s acquisition GAO continues to find that newer programs are demonstrating higher levels of policy and acquisition reforms. GAO knowledge at key decision points, but most are still not fully adhering to a also compiled one- or two-page knowledge-based acquisition approach, putting them at a higher risk for cost assessments of 71 weapon programs. growth and schedule delays. For the programs GAO assessed in depth, GAO These programs were selected based found that a lack of technology maturity, changes to requirements, increases on their cost, stage in the acquisition in the scope of software development, and a lack of focus on reliability were process, and congressional interest. all characteristics of programs that exhibited poorer performance outcomes. DOD disagreed with GAO’s use of total program cost growth as a Last year GAO reported that DOD had begun to incorporate acquisition performance metric because it reforms that require programs to invest more time and resources at the includes costs associated with beginning of the acquisition process refining concepts through early systems capability upgrades and quantity engineering and building prototypes before beginning system development. increases. GAO believes it remains a Many, but not all, planned acquisition programs are adopting these practices. meaningful metric and that the report As GAO has previously recommended, more consistently applying a explicitly accounts for the cost effect knowledge-based approach, as well as improving implementation of of quantity changes. acquisition reforms, can help DOD achieve better outcomes for its portfolio of major weapon system programs. View GAO-11-233SP or key components. For more information, contact Michael J. Sullivan at (202) 512-4841 or [email protected]. United States Government Accountability Office

Contents

Foreword 1

Letter 4 Observations on DOD’s 2010 Major Defense Acquisition Program Portfolio 6 Observations on Factors That Can Affect Program Outcomes 14 Observations from Our Assessment of Knowledge Attained by Key Junctures in the Acquisition Process 18 Observations about DOD’s Implementation of Recent Acquisition Reforms 26 Assessments of Individual Programs 30 Two-Page Assessments of Individual Programs Advanced Extremely High Frequency (AEHF) Satellite 33 AGM-88E Advanced Anti-Radiation Guided Missile (AARGM) 35 Apache Block III (AB3) 37 Army Integrated Air and Missile Defense (Army IAMD) 39 B-2 Extremely High Frequency (EHF) SATCOM Capability, Increment 1 41 BMDS: Ground-Based Midcourse Defense (GMD) 43 BMDS: Terminal High Altitude Area Defense (THAAD) 45 Broad Area Maritime Surveillance (BAMS) Unmanned Aircraft System (UAS) 47 C-5 Reliability Enhancement and Reengining Program  (C-5 RERP) 49 C-130 Avionics Modernization Program (C-130 AMP) 51 CH-53K - Heavy Lift Replacement 53 CVN 21 Future Aircraft Carrier 55 DDG 1000 Destroyer 57 E-2D Advanced Hawkeye (E-2D AHE) 59 Excalibur Precision Guided Extended Range Artillery  Projectile 61 Expeditionary Fighting Vehicle (EFV) 63 F-35 Lightning II (Joint Strike Fighter) 65 Family of Advanced Beyond Line-of-Sight Terminals (FAB-T) 67 Global Hawk (RQ-4A/B) 69 Global Positioning System (GPS) IIIA 71 GPS III OCX Ground Control Segment 73 Gray Eagle 75 Increment 1 Early-Infantry Brigade Combat Team (E-IBCT) 77 Intelligent Munitions System-Scorpion 79

Page i GAO-11-233SP Assessments of Selected Weapon Programs

Contents

Joint Air-to-Ground Missile (JAGM) 81 Joint Air-to-Surface Standoff Missile (JASSM) 83 Joint High Speed Vessel (JHSV) 85 Joint Land Attack Cruise Missile Defense Elevated Netted Sensor System (JLENS) 87 Joint Precision Approach and Landing System (JPALS) 89 Airborne and Maritime/Fixed Station Joint Tactical Radio System (AMF JTRS) 91 Joint Tactical Radio System (JTRS) Ground Mobile Radios  (GMR) 93 Joint Tactical Radio System (JTRS) Handheld, Manpack, and Small Form Fit (HMS) 95 Littoral Combat Ship (LCS) 97 Littoral Combat Ship-Mission Modules 99 LHA Replacement Amphibious Assault Ship 101 Maritime Prepositioning Force (Future) / Mobile Landing  Platform 103 Mobile User Objective System (MUOS) 105 Navy Multiband Terminal (NMT) 107 P-8A Poseidon 109 PATRIOT/Medium Extended Air Defense System (MEADS) Combined Aggregate Program (CAP) Fire Unit 111 Reaper Unmanned Aircraft System 113 Ship to Shore Connector (SSC) 115 Small Diameter Bomb (SDB), Increment II 117 Space Based Infrared System (SBIRS) High Program 119 Standard Missile-6 (SM-6) Extended Range Active Missile  (ERAM) 121 Vertical Take-off and Landing Tactical Unmanned Aerial Vehicle (VTUAV) 123 Virginia Class Submarine (SSN 774) 125 Warfighter Information Network-Tactical (WIN-T)  Increment 2 127 Warfighter Information Network-Tactical (WIN-T)  Increment 3 129 One-Page Assessments of Individual Programs Air and Missile Defense Radar (AMDR) 131 B-2 Defensive Management System (DMS) Modernization 132 B-2 Extremely High Frequency (EHF) SATCOM Capability,  Increment 2 133 BMDS: Airborne Laser Test Bed (ALTB) 134 BMDS: Flexible Target Family (FTF) 135

Page ii GAO-11-233SP Assessments of Selected Weapon Programs

Contents

C-27J Joint Cargo Aircraft (JCA) 136 DDG 51 Destroyer 137 Defense Weather Satellite System (DWSS) 138 Enhanced Polar System (EPS) 139 F-22A Raptor 140 H-1 Upgrades (UH-1Y/AH-1Z) 141 Joint Light Tactical Vehicle (JLTV) 142 KC-X Program 143 National Polar-orbiting Operational Environmental Satellite System (NPOESS) 144 Navy Unmanned Combat Air System Aircraft Carrier Demonstration (UCAS-D) 145 Nett Warrior Increment I 146 Ohio-Class Replacement (OR) / Sea Based Strategic Deterrent 147 Space Fence 148 Stryker Modernization (SMOD) 149 Three Dimensional Expeditionary Long Range Radar  (3DELRR) 150 V-22 Joint Services Advanced Vertical Lift Aircraft (Osprey) 151 Presidential Helicopter (VXX) 152 Agency Comments and Our Evaluation 153

Appendixes Appendix I: Scope and Methodology 157 Appendix II: Changes in DOD’s 2010 Portfolio of Major Defense Acquisition Programs over Time 169 Appendix III: Current and Baseline Cost Estimates for DOD’s 2010 Portfolio of Major Defense Acquisition Programs 170 Appendix IV: Knowledge-Based Acquisition Practices 175 Appendix V: Technology Readiness Levels 177 Appendix VI: Comments from the Department of Defense 179 Appendix VII: GAO Contact and Acknowledgments 181

Related GAO Products 184

Page iii GAO-11-233SP Assessments of Selected Weapon Programs

Contents

Tables Table 1: Changes in DOD’s 2010 Portfolio of Major Defense Acquisition Programs over the Past 2 Years 8 Table 2: Changes in Total Acquisition Cost and Program Acquisition Unit Cost for 10 of the Highest-Cost Acquisition  Programs 10 Table 3: Program Office Composition for 44 DOD Programs, as of 2010 17 Table 4: Changes in DOD’s 2010 Portfolio of Major Defense Acquisition Programs over Time 169 Table 5: Current Cost Estimates and Baseline Cost Estimates for DOD’s 2010 Portfolio of Major Defense Acquisition Programs 170

Figures Figure 1: Programs Meeting DOD, OMB, and GAO Cost Performance Metrics 11 Figure 2: Change in Planned Quantities and Program Acquisition Unit Cost for the F-22 Raptor and DDG 51 Destroyer 13 Figure 3: Relationship between Key Performance Parameter Changes, Research and Development Cost Growth, and Delays in Achieving Initial Operational Capabilities 15 Figure 4: DOD’s Acquisition Cycle and GAO Knowledge Points 19 Figure 5: Implementation of Knowledge-Based Practices by a Program Beginning Engineering and Manufacturing Development since 2009 22 Figure 6: Implementation of Knowledge-Based Practices by Programs Holding Their Critical Design Review since 2009 23 Figure 7: Implementation of Knowledge-Based Practices by Programs Holding Their Production Decision since  2009 25 Figure 8: Depiction of Notional Weapon System Knowledge as Compared with Knowledge-Based Practices 31

Page iv GAO-11-233SP Assessments of Selected Weapon Programs

Contents

Abbreviations

BMDS Ballistic Missile Defense System DAMIR Defense Acquisition Management Information Retrieval DOD Department of Defense IAMD Integrated Air and Missile Defense MDA Missile Defense Agency NA not applicable OMB Office of Management and Budget RDT&E research, development, test, and evaluation SAR Selected Acquisition Report TRL Technology Readiness Level

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.

Page v GAO-11-233SP Assessments of Selected Weapon Programs

Comptroller General A of the United States United States Government Accountability Office Washington, D.C. 20548

March 29, 2011 Letter

Congressional Committees

I am pleased to present GAO’s ninth annual assessment of selected weapon programs. This report provides a snapshot of how well the Department of Defense (DOD) is planning and executing its major defense acquisition programs—an area that is on GAO’s high-risk list.1 It comes at a time when DOD is pressing forward with implementation of the weapon acquisition reforms put in place over the past few years and searching for efficiencies that will allow it to instill fiscal discipline into weapon programs and obtain better buying power for the warfighter and taxpayer. These reforms and efficiency initiatives are consistent with the recommendations we have made over the years emphasizing the need for DOD to acquire greater knowledge about programs’ requirements, technology, and design before they start; improve the realism of cost estimates for both ongoing and new programs; and achieve a balanced mix of weapon systems that are affordable, feasible, and provide the best military value to the warfighter.

To its credit, DOD has demonstrated a strong commitment, at the highest levels, to address the management of its weapon system acquisitions, and the department has started to reprioritize and rebalance its weapon system investments. Since 2009, the Secretary of Defense has proposed canceling or significantly curtailing weapon programs, such as the Army’s Future Combat System, which he characterized as too costly or no longer relevant for current operations. Congress’s support for several of the recommended terminations indicates a willingness to make difficult choices on individual weapon systems and DOD’s major defense acquisition program portfolio as a whole.

The focus of this year’s report is slightly different than in years past. We still provide information on the cumulative cost growth experienced by the 98 programs in DOD’s 2010 portfolio of major defense acquisition programs, but much of our cost analysis examines program performance over the last 2 years. This allows us to better focus on the department’s management of its major defense acquisition programs since key acquisition reforms were put into place by Congress and DOD. In addition, our cost analysis now explicitly accounts for cost growth associated with changes in weapon system quantities, identifies the programs that drive most of DOD’s cost

1GAO, High-Risk Series: An Update, GAO-11-278 (Washington, D.C.: Feb. 16, 2011).

Page 1 GAO-11-233SP Assessments of Selected Weapon Programs

growth, and assesses DOD against cost performance goals agreed to by DOD, the Office of Management and Budget (OMB), and GAO.

Our review this year indicates that DOD is making decisions that put most new or planned programs in a better position to succeed, but still faces challenges to effectively managing its current weapon system programs. DOD can help assure it delivers the promised return on investment for its weapon system spending by using the knowledge-based acquisition approach that is now embodied in law and policy. Our review this year found continued improvement in the knowledge DOD officials had about programs’ technologies, designs, and manufacturing processes for programs that recently progressed through key points in the acquisition process. However, most programs are still proceeding with less knowledge than best practices suggest, putting them at higher risk for cost growth and schedule delays. In fact, a majority of DOD’s major defense acquisition programs did not meet cost performance goals agreed to by DOD, OMB, and GAO for total cost growth over 2-year and 5-year periods and from their initial program estimates. More consistently applying a knowledge- based approach, as well as improving implementation of acquisition reforms, can help DOD achieve better outcomes for its portfolio of major weapon system programs.

While recent acquisition reforms have put newer programs in a better position to field capabilities on-time and at the estimated cost, existing programs such as the Joint Strike Fighter—which began without following knowledge-based acquisition strategies—continue to drive poor outcomes for DOD’s major defense acquisition program portfolio. Over the last 2 years, the total acquisition cost of the programs in DOD’s 2010 portfolio has grown by $135 billion (in fiscal year 2011 dollars). About half of this growth can be associated with quantity changes, versus poor management and execution problems, but at least $70 billion is indicative of such problems. The cost growth on the Joint Strike Fighter alone accounted for almost $34 billion of that amount. Cost overruns of this magnitude on programs that have already spent years in development can only be meaningfully offset by reductions in planned capabilities or quantities. Serious consideration by DOD of these types of tradeoffs will be essential to getting cost growth under control.

This report provides insights that will help DOD place programs in a better position to succeed, thus helping the department maximize its investments. The current acquisition reform environment, coupled with fiscal imperatives, constitute an opportunity to leverage the lessons of the past

Page 2 GAO-11-233SP Assessments of Selected Weapon Programs

and manage risks differently. This environment is shaped by significant acquisition reform legislation, constructive changes in DOD’s acquisition policy, and initiatives by the administration, including making difficult decisions on individual weapon systems. To sustain momentum and make the most of this opportunity, it will be essential that decisions to approve and fund acquisitions be consistent with the reforms and policies aimed at achieving better outcomes. Such decisions will need the support of both DOD and Congress.

Gene L. Dodaro Comptroller General of the United States

Page 3 GAO-11-233SP Assessments of Selected Weapon Programs

A United States Government Accountability Office Washington, D.C. 20548

March 29, 2011 Letter

Congressional Committees

This is GAO’s ninth annual assessment of selected Department of Defense (DOD) weapon programs and the third in response to the mandate in the joint explanatory statement to the DOD Appropriations Act, 2009.1 This report provides a snapshot of how well DOD is planning and executing its weapon programs—an area that is on GAO’s high-risk list. Congress and DOD have long explored ways to improve the acquisition of major weapon systems, yet poor program outcomes persist. In the past 2 years, we have reported improvements in the knowledge programs attained about technologies, design, and manufacturing processes at key points during the acquisition process. However, we have found that most programs continue to proceed with less knowledge than recommended, putting them at higher risk for cost growth and schedule delays. Recent reforms place additional emphasis on applying knowledge-based acquisition practices, which, if implemented, put programs in a better position to field capabilities on time at the estimated cost.

This report includes (1) observations on the performance of DOD’s portfolio of 98 major defense acquisition programs, (2) data on factors, such as performance requirements and software management, that can affect program outcomes, (3) our assessment of the knowledge attained by key junctures in the acquisition process for a subset of 40 weapon programs—primarily in development or the early stages of production— from the 2010 portfolio, and (4) observations on the extent to which DOD is implementing recent acquisition reforms.

There are three sets of programs on which our observations are primarily based in this report. We assessed all 98 major defense acquisition programs for our analysis of portfolio performance; 40 programs in development or early production for our analysis of factors that can affect outcomes and knowledge attained by key junctures; and 14 planned major defense acquisition programs for our analysis of DOD’s progress in implementing selected acquisition reforms. To develop our observations on the

1See Explanatory Statement, 154 Cong. Rec. H 9427, 9526 (daily ed., Sept. 24, 2008), to the Department of Defense Appropriations Act Fiscal Year 2009, contained in Division C of the Consolidated Security, Disaster Assistance, and Continuing Appropriations Act, 2009, Pub. L. No. 110-329.

Page 4 GAO-11-233SP Assessments of Selected Weapon Programs

performance of DOD’s portfolio of 98 major defense acquisition programs, we obtained cost, schedule, and quantity data from DOD’s Selected Acquisition Reports (SAR) and from the Defense Acquisition Management Information Retrieval Purview system.2 For unit cost reporting, DOD breaks several of these 98 programs into components. Therefore, some of our analysis is based on 100 or 101 programs or components. To analyze factors that can affect program outcomes and to assess how well programs are adhering to a knowledge-based acquisition approach, we examined a subset of 40 major defense acquisition programs and components from DOD’s 2010 portfolio that were in development or early production as of June 2010. We submitted a questionnaire to program offices to collect information on aspects of program management including performance requirements, manufacturing planning, software development, and program office staffing. All 40 major defense acquisition programs we surveyed that were in development or early production responded to our questionnaire. We also obtained information on the extent to which they follow knowledge-based practices for technology maturity, design stability, and production maturity using a data collection instrument provided to 40 programs. To examine the extent to which DOD is implementing recent acquisition reforms, we used additional information from a questionnaire submitted to 17 planned major defense acquisition programs approaching program start; 14 of these planned programs responded. In addition to our observations, we present one- or two-page assessments of 71 weapon programs. We chose these 71 programs based on their estimated cost, stage in the acquisition process, and congressional interest.

We conducted this performance audit from June 2010 to March 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings based on our audit objectives. Appendix I contains detailed information on our scope and methodology.

2Major defense acquisition programs are those identified by DOD that require eventual total research, development, test, and evaluation (RDT&E) expenditures, including all planned increments, of more than $365 million, or procurement expenditures, including all planned increments, of more than $2.19 billion, in fiscal year 2000 constant dollars.

Page 5 GAO-11-233SP Assessments of Selected Weapon Programs

Observations on DOD’s Since 2008, the number of programs in DOD’s portfolio of major defense acquisition programs has increased from 96 to 98, and DOD’s total planned 2010 Major Defense investment in these programs has increased by $45 billion to $1.68 trillion.3 Acquisition Program Thirteen programs with a total estimated cost of $174 billion left the Portfolio portfolio.4 Fifteen programs with an estimated cost of $77 billion entered the portfolio.5 These programs are smaller, on average, than those already in the portfolio. Our analysis of the 98 programs in DOD’s 2010 portfolio of major defense acquisition programs allows us to make five observations about the overall portfolio, as well as about the performance of individual programs. First, the total cost of the programs in DOD’s portfolio has grown by about $135 billion, or 9 percent, over the last 2 years, of which about $70 billion cannot be attributed to quantity changes. We focused on this 2-year period instead of program performance against initial estimates in order to assess the department’s recent management of major defense acquisition programs.6 Second, over half of the portfolio’s total cost growth over the last 2 years is driven by 10 of DOD’s largest programs, which are all in production. As these programs leave the portfolio through completion or cancellation, their cost will leave with them. Third, about half of the programs in the portfolio have experienced cost increases that exceed cost

3All dollar figures are in fiscal year 2011 constant dollars unless otherwise noted.

4The 13 programs that have left the portfolio include the Advanced Deployable System (AN/WQR-3), Armed Reconnaissance Helicopter, Defense Integrated Military Human Resources System, Extended Range Munition, Future Combat System, Advanced Anti-Tank Weapon System-Medium (Javelin), Minuteman III Guidance Replacement Program, Mission Planning System, Small Diameter Bomb Increment 1, Ship Self Defense System, Ohio Class SSGN Conversion, T-45TS Naval Undergraduate Jet Flight Training System (Goshawk), and Presidential Helicopter Replacement (VH-71) Program.

5The 15 programs that have entered the portfolio include the Airborne Signals Intelligence Payload-Baseline, Army Integrated Air and Missile Defense, Broad Area Maritime Surveillance Unmanned Aircraft System, C-27J Joint Cargo Aircraft, EA-6B Improved Capability III, Gray Eagle Unmanned Aircraft System, Global Positioning System IIIA, Increment 1 Early-Infantry Brigade Combat Team, Integrated Defensive Electronic Countermeasures, Joint High Speed Vessel, Joint Precision Approach and Landing System, Airborne and Maritime/Fixed Station Joint Tactical Radio System, Predator Unmanned Aircraft System, Reaper Unmanned Aircraft System, and Warfighter Information Network- Tactical Increment 3. Not all of these programs are new starts. Several began as acquisition category II programs before growing in cost.

6We chose a 2-year period instead of a 1-year period because DOD did not issue annual, comprehensive Selected Acquisition Reports in December 2008.

Page 6 GAO-11-233SP Assessments of Selected Weapon Programs

performance goals agreed to by DOD, OMB, and GAO.7 Fourth, almost 80 percent of the programs in the portfolio have experienced an increase in unit cost when compared to their original estimates, thereby reducing DOD’s buying power on these programs. Fifth, on average, the majority of cost growth materialized after programs entered production, meaning they continued to experience significant changes well after the programs and their costs should have stabilized. Additional details about each of these observations follow.

• The total cost of DOD’s 2010 portfolio of major defense acquisition programs has grown by $135 billion, or 9 percent, over the past 2 years, of which about $70 billion cannot be attributed to changes in quantities of some weapon systems. As shown in table 1, since 2008, the total cost to develop and procure all of the weapon systems in this year’s portfolio has increased by $135 billion, or 9 percent.8 About $65 billion of that growth can be attributed to quantity changes on 57 programs.9 The DDG 51 Destroyer, Joint Mine Resistant Ambush Protected vehicle, and C-17A aircraft programs experienced the largest cost increases due to increased quantities and account for almost half of that growth. The remaining $70 billion in cost growth is not attributable to quantity changes. For example, the procurement cost of the Joint Strike Fighter program increased by $28 billion in the last 2 years without a change in quantities. This type of cost growth could be indicative of production problems and inefficiencies or flawed initial cost estimates. Additionally, research and development costs increased by $15 billion over the past 2 years. Five programs—the Joint Strike Fighter, CH-53K Heavy Lift Replacement, F-22 Raptor, Space Based Infrared System High, and Advanced Extremely High Frequency Satellite—accounted for 70 percent of this increase with each

7DOD, OMB, and GAO agreed upon outcome metrics designed to evaluate program performance by measuring acquisition cost growth over the last year, the last 5 years, and since their original program estimate.

8App. II includes 5-year and baseline estimate comparisons for the overall 2010 portfolio.

9Programs that increased quantities had $87 billion in cost growth attributable to those increased purchases and programs that reduced quantities had $22 billion in cost decreases attributable to those reduced purchases. To calculate the portion of the procurement cost growth attributable to quantity changes, we compared a program’s average procurement unit cost from December 2007 with its average procurement unit cost from December 2009. When quantities changed, we multiplied the change by the previous average procurement unit cost to determine the cost growth due to these quantity changes. See app. I for additional information on our scope and methodology.

Page 7 GAO-11-233SP Assessments of Selected Weapon Programs

experiencing research and development cost growth of over $1 billion. With the exception of CH-53K, these programs are all in production, but are still incurring additional research and development costs. For the most part, these programs began with unrealistic business cases, which contributed to these poor outcomes.10

Table 1: Changes in DOD’s 2010 Portfolio of Major Defense Acquisition Programs over the Past 2 Years

Fiscal year 2011 dollars in billions Estimated portfolio Estimated portfolio Estimated portfolio cost growth since Percentage growth cost in 2008a cost in 2010 2008b since 2008b Total estimated research and $407 $428 $15 5% development cost Total estimated procurement cost 1,089 1,219 121 11 Total estimated acquisition cost 1,531 1,680 135 9 Average delay in delivering initial – – 5 months 8 capabilities Source: GAO analysis of DOD data. Notes: In addition to research and development and procurement costs, total acquisition cost includes acquisition operation and maintenance and system-specific military construction costs. Details on program costs used for this analysis are provided in app. III. aThe 2008 estimate includes costs for the 83 programs that are at least 2 years old, and the first available estimate for the remaining 15 programs that are new to the 2010 portfolio. bThe portfolio cost columns include the reported cost of the Ballistic Missile Defense System (BMDS); however, the portfolio cost growth columns do not include $6 billion in research and development and $9 billion in procurement cost changes for the BMDS. DOD does not consider these adjustments to represent cost growth because the program has been allowed to add 2 years of new funding with each biennial budget.

In addition to higher costs, programs in the 2010 portfolio have also experienced additional delays within the past 2 years. The average delay in delivering initial capabilities increased by 5 months. When examined from a longer-term perspective, the average delay in delivering initial capabilities is 22 months for these programs when measured against their first full estimates. See appendix II for our analysis of cost growth and delays in

10GAO, Tactical Aircraft: Recapitalization Goals Are Not Supported by Knowledge-Based F-22A and JSF Business Cases, GAO-06-487T (Washington, D.C.: Mar. 16, 2006); Defense Acquisitions: Despite Restructuring, SBIRS High Program Remains at Risk of Cost and Schedule Overruns, GAO-04-48 (Washington, D.C.: Oct. 31, 2003); and Defense Acquisitions: Space System Acquisition Risks and Keys to Addressing Them, GAO-06-776R (Washington, D.C.: June 1, 2006).

Page 8 GAO-11-233SP Assessments of Selected Weapon Programs

delivering initial capabilities against first full estimates for DOD’s 2010 portfolio of major defense acquisition programs.

• Ten of DOD’s largest acquisition programs account for over half the portfolio’s total acquisition cost growth over the past 2 years. DOD’s largest programs represent more than half of its total investment in major defense acquisition programs. These programs range in age from 3 to 32 years; all 10 are in production; and 8 have attained initial operational capability. Collectively, the estimated cost of these programs is $853 billion. These 10 programs account for $79 billion in cost growth over the last 2 years, or over half of the $135 billion in total cost growth for this period. Of the $79 billion in cost growth, $35 billion is attributable to increased purchases—primarily of the C-17A aircraft, DDG 51 Destroyer, and Joint Mine Resistant Ambush Protected family of vehicles, which has been in high demand because of the conflicts in Iraq and Afghanistan. When examined from a more historical perspective, the total acquisition cost of these 10 programs has grown by $196 billion over their first full estimates, which is almost half of the $402 billion in total cost growth for the 2010 portfolio. As these programs leave the portfolio through completion or cancellation, their cost will leave with them. While some of these programs are nearing the end of their procurement, others such as the Joint Strike Fighter, Virginia Class Submarine, V-22, and CVN 21 will continue to demand large amounts of annual funding. Table 2 provides a summary of 10 of the largest DOD acquisition programs.

Page 9 GAO-11-233SP Assessments of Selected Weapon Programs

Table 2: Changes in Total Acquisition Cost and Program Acquisition Unit Cost for 10 of the Highest-Cost Acquisition Programs

Total acquisition cost Program acquisition unit cost (fiscal year 2011 dollars in millions) (fiscal year 2011 dollars in millions) Percentage Change over change over Program 2008 estimate 2010 estimate the last 2 years 2008 estimate 2010 estimate the last 2 years Joint Strike Fighter 249,690 283,674 33,984 101.7 115.5 13.6 DDG 51 Destroyer 77,382 94,344 16,961 1,248.1 1,328.8 6.5 C-17A Globemaster III 75,046 82,347 7,301 395.0 369.3 -6.5 Virginia Class 83,194 82,193 -1,002 2,773.1 2,739.8 -1.2 Submarine (SSN 774) F-22 Raptor 75,200 77,393 2,193 408.7 411.7 0.7

V-22 Joint Services 56,659 56,061 -598 123.7 122.4 -1.1 Advanced Vertical Lift Aircraft (Osprey) F/A-18E/F Super 52,824 54,625 1,801 107.1 106.1 -1.0 Hornet Trident II Missile 50,611 51,410 799 90.2 91.6 1.6 Joint Mine Resistant 22,792 36,375 13,583 1.5 1.6 7.2 Ambush Protected (MRAP) CVN 21 Future Aircraft 30,513 34,186 3,673 10,171.0 11,395.2 12.0 Carrier Total 773,911 852,607 78,696 – – – Source: GAO analysis of DOD data. Note: Figures may not add due to rounding.

• Fewer than half of the programs in DOD’s 2010 portfolio are meeting established performance metrics for cost growth. In December 2008, DOD, working with OMB and GAO, developed a set of outcome metrics and goals to measure program performance over time. As shown in figure 1, less than half of the programs in DOD’s 2010 portfolio met the total acquisition cost growth goals that were set.11 The metrics also show that those programs experiencing significant cost growth—more than 15 percent over initial estimates—dominate DOD’s

11These metrics are designed to capture total cost growth performance over 1-year and 5- year periods and from the original program estimate. We modified these metrics to measure a 2-year comparison since cost data were not available to make a 1-year comparison.

Page 10 GAO-11-233SP Assessments of Selected Weapon Programs

portfolio, representing about 72 percent of DOD’s total investment in its major defense acquisition programs.12

Figure 1: Programs Meeting DOD, OMB, and GAO Cost Performance Metrics 100

90

80 44 28% 49 49 70

60

50

40

30 57 72% 51 51 20

10

0 2-year 5-year Baseline Percent of DOD’s total comparison comparison comparison investment in major <4% <10% <15% defense acquisition growth growth growth programs

Programs that meet criteria

Programs that do not meetcriteria

Source: GAO analysis of DOD data. Notes: The number of programs represents those in the 2010 portfolio—those with December 2009 SARs—which break down several programs into smaller elements for reporting purposes. One program, Airborne Signals Intelligence Payload (ASIP)-Baseline, was not included in 2-year and 5-year comparisons because data were not available to make those comparisons.

• DOD’s buying power has been reduced for almost 80 percent of its portfolio of major defense acquisition programs. Of the 100 programs or components in DOD’s 2010 portfolio that reported program acquisition unit cost data, 79 are planning to deliver capabilities at

12The original estimates we used are primarily programs’ cost estimates at development start. These may differ from DOD baseline estimates used for Nunn-McCurdy cost growth purposes, which can be reset after a Nunn-McCurdy unit cost breach of the critical threshold.

Page 11 GAO-11-233SP Assessments of Selected Weapon Programs

higher unit costs than originally estimated, while only 21 are planning to deliver capabilities at or below initial estimates.13 We did not examine whether these programs delivered a lower or higher level of performance than initially promised. To quantify the change in DOD’s buying power, we examined changes in program acquisition unit cost and quantities.14 An example of a program that experienced reduced buying power is the F-22 Raptor. Despite a 70 percent reduction in quantities for the program, total acquisition costs have only decreased by 14 percent, due to research and development and average procurement unit cost increases. As a result, program acquisition unit costs for the F-22 Raptor have almost tripled, from $139 million to $412 million per airplane. For the current 188 aircraft program, the $273 million increase per plane translates to $51.3 billion in lost buying power for the F-22 program as a whole. Conversely, some programs are planning to deliver more quantities than planned at a lower program acquisition unit cost, which translates into increased buying power for DOD. For example, despite research and development costs almost tripling and the total program cost increasing by $79 billion, the DDG 51 Destroyer’s program acquisition unit cost has decreased by about 20 percent, because it has spread out those research and development costs over significantly higher quantities and its average procurement unit cost has decreased over time. For the currently planned fleet of 71 ships, the $333 million reduction per ship corresponds to $24 billion in increased buying power for the program as a whole.

13Program acquisition unit cost is the total cost for development, procurement, acquisition operation and maintenance, and system-specific military construction for the acquisition program divided by the number of items to be produced. DOD’s 2010 portfolio includes 98 programs with SARs; however, DOD’s SAR summary tables break down several of these programs into smaller elements. We did not include the Missile Defense Agency’s (MDA) Ballistic Missile Defense System because comparable cost and schedule data were not available, or the National Polar-orbiting Operational Environmental Satellite System, because quantities were reduced to zero.

14We calculated the effect on DOD’s buying power from a program by multiplying the change in the program acquisition unit cost by the current planned quantities.

Page 12 GAO-11-233SP Assessments of Selected Weapon Programs

Figure 2: Change in Planned Quantities and Program Acquisition Unit Cost for the F-22 Raptor and DDG 51 Destroyer F-22 Raptor DDG 51 Destroyer

Number of units Dollars (in millions) Number of units Dollars (in millions) 700 700 90 1,800 648 $1,662 80 1,600 600 600 71 70 $1,329 1,400 500 500 60 1,200 $412 400 400 50 1,000

300 300 40 800

30 600 200 188 200 $139 20 400

100 100 9 10 200

0 0 0 0 Baseline Current Baseline Current estimate estimate estimate estimate

Total quantity

Program acquisition unit cost

Source: GAO analysis of DOD data.

• On average, the majority of cost growth materialized after programs entered production, meaning they continued to experience significant changes well after the programs and their costs should have stabilized. For the 56 major defense acquisition programs in DOD’s 2010 portfolio that had production cost estimates as of the end of 2009, we found that the average program experienced the majority of its research and development and procurement cost growth after its production decision. Fifty-two percent of the average program’s research and development cost growth was incurred after production start. Additionally, 65 percent of the average program’s procurement cost growth materialized after production start. On average, these programs are 14 years old. Given the age of the programs in this group, most of them were started before DOD acquisition polices were revised to promote a knowledge-based, evolutionary acquisition approach. Therefore, newer programs that follow a knowledge-based approach

Page 13 GAO-11-233SP Assessments of Selected Weapon Programs

may be better positioned to avoid cost growth this late in the acquisition cycle.

Observations on For 40 individual weapon programs in DOD’s 2010 portfolio, we collected and assessed data on DOD’s management of requirements, software, and Factors That Can program office staffing. We previously reported that both requirements Affect Program changes and increases in the scope of software development were Outcomes associated with poor program outcomes.15 We found similar results this year. Our analysis of the data allows us to make three observations. First, over half of the programs in our assessment made a change to a key performance requirement after development start and experienced higher levels of cost growth and longer schedule delays than programs with unchanged requirements. Second, half of the programs in our assessment that provided data on software lines of code saw the scope of software development increase after development start, which also corresponded with poorer outcomes. Third, programs continue to use contractors to make up for staffing shortfalls, though programs’ reliance on nongovernment personnel has declined from last year’s assessment. Additional details about each of these observations follow.

• Programs that modified key performance requirements after development start experienced higher levels of cost growth and longer delays in delivering capabilities. Of the 39 programs in our current assessment that reported tracking requirements changes since development start, 21 reported having had at least one change to a key performance parameter—a top-level requirement. Specifically, 10 of the 21 programs reported adding or enhancing a key performance parameter; 3 of the 21 programs reported reducing, deferring, or deleting a key performance parameter; and 8 of the 21 programs reported making both types of changes to key performance parameters. Most of the programs that experienced requirements changes are programs that started prior to 2005. While changing requirements creates instability and, therefore, can adversely affect program outcomes, it is also possible that some programs experiencing poor outcomes may be decreasing program requirements in an effort to prevent further cost growth. As shown in figure 3, programs with changes to performance requirements experienced roughly four times

15GAO, Defense Acquisitions: Assessments of Selected Weapon Programs, GAO-09-326SP (Washington, D.C.: Mar. 30, 2009).

Page 14 GAO-11-233SP Assessments of Selected Weapon Programs

more growth in research and development costs and three to five times greater schedule delays compared to programs with unchanged requirements. Similarly, programs with increases to key system attributes—lower level, but still crucial requirements of the system— experienced greater, albeit less pronounced, cost growth and schedule delays than other programs.

Figure 3: Relationship between Key Performance Parameter Changes, Research and Development Cost Growth, and Delays in Achieving Initial Operational Capabilities

Percentage 100

90 84

80 75

70

60

50

40 40

30 27

20 18

10 8

0 Programs with Programs with Programs with decreased, deferred, new or enhanced no change in or deleted requirements requirements requirements (18 programs) (18 programs) (11 programs)

Average increase in research and development costs

Average increase in length of acquisition cycle

Source: GAO analysis of DOD data.

Notes: Programs that had both increases and decreases in key performance parameters are included in both categories. Cost and schedule data were not available for the Joint Air-to-Surface Standoff Missile Extended Range variant, which had a new or enhanced requirement.

• Substantial increases in the scope of software development efforts after development start also correspond to higher cost growth and longer schedule delays. In our last several assessments,

Page 15 GAO-11-233SP Assessments of Selected Weapon Programs

we reported that programs experiencing more growth in software lines of code since development start had higher development cost growth and longer schedule delays than other programs. Similarly, for the 25 programs in our current assessment that reported data on lines of code, we found that increases in total lines of code after development start correlate highly with both increases in research and development costs and longer delays in achieving initial operational capability. Over half of these programs, or 14 of 25, estimated that the number of lines of code required for the system to function has grown or will grow by 25 percent or more. These programs tend to be those that began development more than 5 years ago. Newer programs have also experienced some software growth, though it has been less severe, on average. In addition to measuring growth in software lines of code, we have previously reported that collecting earned value management data for software development is a good management practice. Thirty-two of the 40 programs in our assessment reported collecting such data to help manage software development by allowing visibility into schedule and cost performance of software. These programs generally had software efforts that were more than twice as large, on average, as those programs that reported not collecting earned value data for software development. Finally, we have previously reported that tracking and capturing software defects in-phase is important because discovering defects out of phase can cause expensive rework later in programs. For the 21 programs that reported collecting some type of software defect data, on average, only 69 percent of the defects were corrected in the phase of software development in which they occurred.

• Programs continue to use contractors to make up for staffing shortfalls, but reliance on nongovernment personnel has decreased. Congress and DOD have made it a priority to ensure the acquisition workforce has the capacity, personnel, and skills needed to properly perform its mission. Most programs, however, continue to struggle to fill all staff positions authorized. Specifically, 36 of the 44 programs we surveyed reported having been authorized all the positions they requested, but a majority—23 programs—were unable to fill all of them.16 A majority of programs we assessed reported that they are in the process of staffing unfilled positions. Several cited delays and difficulty in finding qualified candidates as reasons for not being able to fill these

16In addition to data from 40 major defense acquisition programs, our analysis of program staffing includes data from four MDA elements.

Page 16 GAO-11-233SP Assessments of Selected Weapon Programs

positions. Additionally, almost all programs—39 of 44—reported using support contractors to make up for shortfalls in government personnel or capabilities.  Program offices’ reliance on contractors has decreased since last year’s assessment. As shown in table 3, more than half (55 percent) of all program office staff for the 44 programs in our assessment were government personnel—a reversal of the downward trend in the percentage of government personnel that we have reported in the previous 3 years.17 The percentage of support contractors declined in every discipline. Support contractors still fill the majority of administrative support positions; however, the greatest numbers of support contractors continue to be in engineering and technical positions.

Table 3: Program Office Composition for 44 DOD Programs, as of 2010

Program Business Administrative Percentage of staff management Engineering Contracting functions support Other Total Military personnel 30 7 6 3 8 11 10 Civilian government 43 45 84 57 30 25 46 Total government 73 52 90 60 37 36 55 Support contractors 27 40 10 38 61 64 39 Other nongovernmenta 0 8 0 2 2 0 6 Total nongovernment 27 48 10 40 63 64 45 Source: GAO analysis of DOD data. Note: Totals may not add due to rounding. aOther nongovernment includes federally funded research and development center and university- affiliated employees.

17We reported last year that 49 percent of program office staff were government personnel. The program offices we collected data from differ year to year. However, in both years, we focused on programs that were in development or the early stages of production.

Page 17 GAO-11-233SP Assessments of Selected Weapon Programs

Observations from Our Good acquisition outcomes require the use of a knowledge-based approach to product development that demonstrates high levels of knowledge before Assessment of significant commitments are made. In essence, knowledge supplants risk Knowledge Attained by over time. In our past work examining weapon acquisition issues and best Key Junctures in the practices for product development, we have found that leading commercial firms pursue an acquisition approach that is anchored in knowledge, Acquisition Process whereby high levels of product knowledge are demonstrated by critical points in the acquisition process.18 On the basis of this work, we have identified three key knowledge points during the acquisition cycle— development start; design review, which occurs during engineering and manufacturing development; and production start—at which programs need to demonstrate critical levels of knowledge to proceed. Figure 4 compares DOD acquisition milestones with the timing of the three knowledge points.

18GAO, Best Practices: Better Management of Technology Development Can Improve Weapon System Outcomes, GAO/NSIAD-99-162 (Washington, D.C.: July 30, 1999); Best Practices: Better Matching of Needs and Resources Will Lead to Better Weapon System Outcomes, GAO-01-288 (Washington, D.C.: Mar. 8, 2001); Best Practices: Capturing Design and Manufacturing Knowledge Early Improves Acquisition Outcomes, GAO-02-701 (Washington, D.C.: July 15, 2002); Defense Acquisitions: A Knowledge-Based Funding Approach Could Improve Major Weapon System Program Outcomes, GAO-08-619 (Washington, D.C.: July 2, 2008); Best Practices: High Levels of Knowledge at Key Points Differentiate Commercial Shipbuilding from Navy Shipbuilding, GAO-09-322 (Washington, D.C.: May 13, 2009); Best Practices: DOD Can Achieve Better Outcomes by Standardizing the Way Manufacturing Risks Are Managed, GAO-10-439 (Washington, D.C: Apr. 22, 2010).

Page 18 GAO-11-233SP Assessments of Selected Weapon Programs

Figure 4: DOD’s Acquisition Cycle and GAO Knowledge Points

Development start Production start DOD acquisition milestones (milestone B) (milestone C)

Engineering and manufacturing development

Technology Integration Demonstration Production Development

Knowledge point 1 Knowledge point 2 Knowledge point 3 Technologies and Design performs Production can meet GAO knowledge points resources match as expected cost, schedule, and requirements quality targets

Source: GAO.

The building of knowledge consists of information that should be gathered at these three critical points over the course of a program:

• Knowledge point 1: Resources and requirements match. Achieving a high level of technology maturity by the start of system development is an important indicator of whether this match has been made. This means that the technologies needed to meet essential product requirements have been demonstrated to work in their intended environment. In addition, the developer has completed a preliminary design of the product that shows the design is feasible.

• Knowledge point 2: Product design is stable. This point occurs when a program determines that a product’s design will meet customer requirements, as well as cost, schedule, and reliability targets. A best practice is to achieve design stability at the system-level critical design review, usually held midway through system development. Completion of at least 90 percent of engineering drawings at this point or 100 percent of the 3D product models for ships at fabrication start provides tangible evidence that the product’s design is stable, and a prototype demonstration shows that the design is capable of meeting performance requirements.

• Knowledge point 3: Manufacturing processes are mature. This point is achieved when it has been demonstrated that the developer can manufacture the product within cost, schedule, and quality targets. A

Page 19 GAO-11-233SP Assessments of Selected Weapon Programs

best practice is to ensure that all critical manufacturing processes are in statistical control—that is, they are repeatable, sustainable, and capable of consistently producing parts within the product’s quality tolerances and standards—at the start of production.

A knowledge-based acquisition approach is a cumulative process in which certain knowledge is acquired by key decision points before proceeding. In other words, demonstrating technology maturity is a prerequisite for moving forward into system development, during which the focus should be on design and integration. Additional details about key practices at each of the knowledge points can be found in appendix IV.

For 40 individual weapon programs in development and early production in DOD’s 2010 portfolio, we assessed the knowledge attained by key junctures in the acquisition process. In particular, we focused on the 17 programs that progressed through these key acquisition points since 2009 and evaluated their adherence to knowledge-based practices. While we continue to find that newer programs are demonstrating higher levels of knowledge at key decision points, most are still not fully adhering to a knowledge-based acquisition approach, putting them at a higher risk for cost growth and schedule delays.19 Only one program in our assessment began system development since 2009, and it did so with all its critical technologies nearing maturity, in accordance with DOD and statutory requirements. However, it did not fully mature its critical technologies before beginning development, in accordance with knowledge-based practices, and only three nonship programs in our assessment had done so by development start. Six of nine programs that held a critical design review since 2009 did so with a stable design; however, these programs did not implement other practices that increase confidence that the design is stable and capable of meeting performance requirements. Finally, almost all programs that held a production decision since 2009 identified key product characteristics and critical manufacturing processes; however, none of the programs demonstrated that critical manufacturing processes were in control and only half tested production-representative prototypes prior to this decision. Additional details about these observations follow.

19Not all programs provided information for every knowledge point or had reached all of the knowledge points—development start, design review, and production start. Because knowledge points differ for shipbuilding programs, we exclude them from our assessment of certain knowledge-based practices. App. IV contains a list of knowledge-based practices at each of the three knowledge points.

Page 20 GAO-11-233SP Assessments of Selected Weapon Programs

• The one program in our assessment that began development since 2009 did so with all its technologies nearing maturity, but few programs overall began development with fully mature technologies. The Army’s Integrated Air and Missile Defense (IAMD) program began development in 2009 with all critical technologies at least nearing maturity—that is, demonstrated in a relevant environment—in accordance with DOD and statutory requirements,20 but did not demonstrate them in a realistic environment as GAO has recommended.21 Our analysis of the 26 nonship programs in our assessment that provided technology data shows that only three of these programs began development with fully mature technologies— that is, demonstrated in a realistic environment. Our analysis also shows that mature technologies are associated with improved program outcomes. Specifically, the 3 programs that began development with fully mature technologies have experienced 4 percent less growth in research and development costs over their first estimates, on average, compared to the 11 programs that began development with all technologies at least nearing maturity, and 33 percent less growth, on average, than the 12 programs with at least one immature technology at the start of system development.  In addition, as shown in figure 5, the IAMD program did not implement other knowledge-based practices before beginning development, including holding a program-level preliminary design review and constraining the length of system development. We have previously reported that before starting development, programs should hold key system engineering events, such as the preliminary design review, to ensure that requirements are defined and feasible and that the proposed design can meet those requirements within cost, schedule, and other 

20According to DOD policy, in order to be considered mature enough to use in product development, technology shall have been demonstrated in a relevant environment or, preferably, in an operational environment. Department of Defense Instruction 5000.02, Operation of the Defense Acquisition System, enc. 2, para. 5.d.(4) (Dec. 8, 2008). In addition, a major defense acquisition program may not receive milestone B approval until the milestone decision authority certifies that the technology in the program has been demonstrated in a relevant environment. National Defense Authorization Act for Fiscal Year 2006, Pub. L. No. 109-163, § 801 (codified as amended at 10 U.S.C. § 2366b(a)(3)(D)).

21Demonstration in a relevant environment is Technology Readiness Level (TRL) 6. Demonstration in a realistic environment is TRL 7. See app. V for a detailed description of TRLs.

Page 21 GAO-11-233SP Assessments of Selected Weapon Programs

system constraints.22 IAMD did hold a system requirements review and a system functional review before development start, which is a major improvement over other programs in our assessment, which held these reviews, on average, 18 months and 25 months after development start, respectively. Knowledge-based acquisition practices also recommend that a system or increment be developed in 5 to 6 years or fewer.23 Further, DOD acquisition policy states that a condition for exiting technology development is that a system or increment can be developed for production within a short time frame, defined as normally less than 5 years for weapons systems. Constraining development time increases the predictability of funding needs and the likelihood of program success. While IAMD plans to follow an incremental approach, system development will take almost 7 years.

Figure 5: Implementation of Knowledge-Based Practices by a Program Beginning Engineering and Manufacturing Development since 2009

Knowledge-based practices at development start IAMD Knowledge point 1

Mature all critical technologies Hold system requirements review Hold system functional review Hold preliminary design review a Constrain development phase to 6 years or less

Practice implemented by program

Practice not implemented by program

Source: GAO analysis of DOD data. aThe IAMD program received a waiver for the requirement to hold a preliminary design review before beginning system development.

22GAO-09-326SP. A major defense acquisition program may not receive milestone B approval until the program has held a preliminary design review and the milestone decision authority has conducted a formal postpreliminary design review assessment and certified on the basis of such assessment that the program demonstrates a high likelihood of accomplishing its intended mission. Weapon Systems Acquisition Reform Act of 2009, Pub. L. No. 111-23, § 205(a)(3) (codified at 10 U.S.C. § 2366b(a)(2)). IAMD received a waiver from this requirement.

23GAO-08-619.

Page 22 GAO-11-233SP Assessments of Selected Weapon Programs

• Six of nine programs in our assessment that held a critical design review since 2009 did so with a stable design; however, these programs did not implement other knowledge-based practices to increase confidence that the design is stable. Knowing a product’s design is stable before system demonstration reduces the risk of costly design changes occurring during the manufacturing of production- representative prototypes—when investments in acquisitions become more significant. The overall design knowledge that programs have demonstrated at their critical design review has increased over the last few years, and six of nine programs that held a design review since 2009 did so with a stable design. However, as shown in figure 6, none of the nine programs in our assessment that held their critical design review since 2009 demonstrated that their design is capable of meeting performance requirements by testing an integrated prototype before the design review. We have previously reported that early system prototypes are useful to demonstrate design stability and that the design will work and can be built. On average, the nine programs tested or plan to test an integrated prototype 13 months after the critical design review, which is an improvement over the 31-month average we reported in last year’s assessment.

Figure 6: Implementation of Knowledge-Based Practices by Programs Holding Their Critical Design Review since 2009

Knowledge-based practices at design review Knowledge point 2

AB3 AMF JTRS CH-53K FAB-T GPS IIIA IncrementE-IBCT 1 JPALS PATRIOT/MEADSFire CAP Unit Reaper Mature all critical technologies Release at least 90 percent of design drawings Test a system-level integrated prototype Use a reliability growth curve Conduct producibility assessments to identify manufacturing risks for key technologies Complete failure modes and effects analysis

Practice implemented by program

Practice not implemented by program

Practice not applicable or information not available

Source: GAO analysis of DOD data.

Page 23 GAO-11-233SP Assessments of Selected Weapon Programs

Many of these programs are also still concurrently developing technologies and finalizing designs, which can lead to cost and schedule inefficiencies and rework. Of the nine programs, only three had fully matured all critical technologies at this point in the acquisition cycle. The remaining programs accepted technologies into their product’s design based on no more than a laboratory demonstration of basic performance, technical feasibility, and functionality instead of a representative model or prototype in a realistic environment.

Despite not demonstrating that their design is stable, many of these programs are taking steps to plan for production. Almost all programs that held their critical design review since 2009 reported completing failure modes and effects analysis to identify potential failures and early design fixes and conducting producibility assessments to identify manufacturing risks for key technologies. However, only three programs reported having a reliability growth curve at the time of the critical design review. Reliability growth testing provides visibility over how reliability is improving and uncovers design problems so fixes can be incorporated before production begins. Our assessment of programs in development or early production that provided cost data shows that on average, programs using a reliability growth curve have experienced about one-third the research and development cost growth after development start than programs that have not used a reliability growth curve.

• Almost all programs that held a production decision since 2009 took steps to plan for manufacturing; however, none of the programs demonstrated that critical manufacturing processes were in control, and only half tested production-representative prototypes prior to this decision. Capturing critical manufacturing knowledge before entering production helps ensure that a weapon system will work as intended and can be manufactured efficiently to meet cost, schedule, and quality targets. Identifying key product characteristics and the associated critical manufacturing processes is a key initial step to ensuring production elements are stable and in control. As shown in figure 7, almost all of the programs in our assessment that made a production decision since 2009 reported conducting these activities before entering production. However, none of the 10 programs that made a production decision since 2009 demonstrated that their critical manufacturing processes were in  

Page 24 GAO-11-233SP Assessments of Selected Weapon Programs

statistical control at production start.24 Bringing processes under statistical control reduces variations in parts manufacturing, thus reducing the potential for defects, and is generally less costly than performing extensive inspection after a product is built.

Figure 7: Implementation of Knowledge-Based Practices by Programs Holding Their Production Decision since 2009

Knowledge-based practices at production decision gle Knowledge point 3

AB3 C-130 AMP E-2D AHE GPS IIIA Gray Ea IncrementE-IBCT 1 NMT P-8A SM-6 WIN-TIncrement 2 Mature all critical technologies Release at least 90 percent of design drawings Identify key product characteristics Identify critical manufacturing processes Demonstrate critical processes are in statistical control Demonstrate critical processes on a pilot production line Test a production-representative prototype

Practice implemented by program

Practice not implemented by program

Practice not applicable or information not available

Source: GAO analysis of DOD data.

Several programs also began production without knowledge they ought to have gained much earlier in their acquisition process. One program— Increment 1 Early-Infantry Brigade Combat Team (E-IBCT)—entered production without mature technologies or a stable design. Another program—Gray Eagle Unmanned Aircraft System—also did not have

24DOD policy states that the knowledge required for a major defense acquisition program to proceed beyond low-rate initial production shall include demonstrated control of the manufacturing process and acceptable reliability, the collection of statistical process control data, and demonstrated control and capability of critical processes. Department of Defense Instruction 5000.02, Operation of the Defense Acquisition System, enc. 2, para. 7.c.(2) (Dec. 8, 2008). Since we focus on the low-rate production decision, we did not specifically assess compliance with this requirement.

Page 25 GAO-11-233SP Assessments of Selected Weapon Programs

mature technologies by its production decision, while the Standard  Missile-6 (SM-6) began production without completing its design. These programs are at risk of continued cost growth.

Additionally, many of these programs are still not testing production- representative prototypes before committing to production. We previously reported that production and postproduction costs are minimized when a fully integrated, capable prototype is demonstrated to show that the system will work as intended in a reliable manner. These benefits are maximized when tests are completed prior to a production decision, because making design changes after production begins can be both costly and inefficient. Moreover, DOD’s December 2008 revision to its acquisition policy requires programs to test production-representative articles before entering production. Only 5 of the 10 programs that held a production decision since 2009 reported testing a production-representative prototype before their production decision. However, 11 of the 13 programs that are scheduled to hold their production decision after 2010 and provided data do plan to test a fully configured prototype before that decision.

Observations about Last year we reported that DOD had begun to incorporate recent acquisition reforms into the strategies of new programs. These reforms—in DOD’s Implementation DOD’s December 2008 revised acquisition policy, the Weapon Systems of Recent Acquisition Acquisition Reform Act of 2009, and recent defense authorization acts— Reforms require programs to invest more time and resources at the beginning of the acquisition process refining concepts through early systems engineering and building prototypes before beginning system development, among other requirements. In addition, for ongoing programs, DOD policy and statute require establishment of annual configuration steering boards to review all program requirements changes as well as to make recommendations on proposed descoping options that could help maintain a program’s cost and schedule targets. Our examination of the extent to which weapon systems are adhering to recent acquisition reforms indicates that many, but not all, programs are implementing these reforms.

Our assessment allows us to make five observations concerning DOD’s progress in implementing legislative and policy reforms. First, almost all of the 14 planned major defense acquisition programs we reviewed intend to hold preliminary design reviews before beginning development, but fewer are taking other actions, such as developing prototypes, that could improve  

Page 26 GAO-11-233SP Assessments of Selected Weapon Programs

their chances of success.25 Second, seven of these programs reported making major cost, schedule, or performance tradeoffs to-date, as required under a DOD and statutory requirement that programs make appropriate tradeoffs before beginning development. Third, six of the planned programs have acquisition strategies that include competition after the start of development. Fourth, of the 40 programs in our assessment that have begun development or are in the early stages of production, about one-third have not yet held a configuration steering board meeting, and only five programs reported presenting descoping options at this meeting. Finally, as part of DOD’s effort to “do more without more,” the Under Secretary of Defense for Acquisition, Technology and Logistics is beginning to implement a range of efficiency initiatives that focus on affordability, tradeoffs, and portfolio reviews, and are consistent with past GAO recommendations. Additional details about these observations follow.

• Almost all of the planned major defense acquisition programs in our assessment intend to conduct a preliminary design review before development start, but fewer are taking other actions, such as developing prototypes, that could improve their chances of success. Thirteen of the 14 planned major defense acquisition programs we reviewed intend to hold a preliminary design review,26 and all 10 that provided dates for this review plan to hold it before milestone B—the beginning of system development—as required by the Weapon Systems Acquisition Reform Act of 2009. Nine of the 14 planned programs intend to develop prototypes of the proposed weapon system or a key system element before milestone B.27 Programs can seek a waiver of the prototyping requirement, as provided by the policy. Holding a preliminary design review before beginning development can help ensure that program requirements are defined and feasible, but by not developing prototypes, some programs might be missing an

25We refer to DOD’s designated pre–major defense acquisition programs (pre-MDAPs) as planned programs throughout this report.

26The Common Vertical Lift Support Platform program does not plan to hold a preliminary design review because the program is planning to enter the acquisition cycle at production start.

27According to DOD acquisition policy, the technology development strategy for a major defense acquisition program shall provide for prototypes of the system or, if a system prototype is not feasible, for prototypes of critical subsystems before the program gets approval to enter development. Under Secretary of Defense, Acquisition, Technology and Logistics Directive-Type Memorandum (DTM) 09-027—Implementation of the Weapon Systems Acquisition Reform Act of 2009, attachment 1, para. 4 (Dec. 4, 2009).

Page 27 GAO-11-233SP Assessments of Selected Weapon Programs

opportunity to further reduce technical risk, refine requirements, validate designs and cost estimates, and evaluate manufacturing processes.

Programs can also put themselves in a better position to succeed by implementing incremental acquisition strategies that limit the time in development and constrain requirements. Seven of the 14 planned programs in our assessment reported using DOD’s preferred incremental, or evolutionary, acquisition approach, while the other 7 programs planned for a single-step-to-full-capability. The single-step-to- full-capability approach affords few, if any, opportunities to incrementally, and thus more quickly and inexpensively, adapt the final system to the changing needs of the warfighter. Conversely, an evolutionary acquisition strategy emphasizes a more flexible approach that can help reduce risk because it delivers the weapon system in more manageable increments. In addition, six of the nine planned programs in our assessment that provided data plan to deliver capabilities in less than 6 years, the recommended time limit for system development, while the remaining three programs are planning for longer development periods. We have previously reported that constraining development cycles increases the predictability of funding needs and the likelihood of program success, while unconstrained and lengthy cycle times lead to higher costs and diminished military effectiveness.

• Half of the planned programs we reviewed reported making major cost, schedule, or performance tradeoffs before beginning development. DOD acquisition policy and statute require that the Milestone Decision Authority certify, before a program begins development, that appropriate tradeoffs among cost, schedule, and performance objectives have been made to ensure that the program is affordable. Seven of the 14 planned programs in our assessment reported making major cost, schedule, or performance tradeoffs during the technology development phase to-date. For example, in an effort to reduce cost, the Mobile Landing Platform program reduced requirements for size, personnel accommodations, and cargo fuel. The Joint Requirements Oversight Council is also statutorily required to consider cost, schedule, and performance tradeoffs when validating joint military requirements.

• Fewer than half of planned programs have acquisition strategies to ensure competition throughout the acquisition cycle. The

Page 28 GAO-11-233SP Assessments of Selected Weapon Programs

Weapon Systems Acquisition Reform Act of 2009 requires that DOD ensure that the acquisition strategy for each major defense acquisition program includes measures to ensure competition, or the option of competition, throughout the program’s life cycle. These measures may include developing competitive prototypes, dual-source contracting, and periodic competitions for subsystem upgrades. Incorporating competition throughout a program’s life cycle can be used to drive productivity and thereby reduce program costs. Six of the 14 planned programs in our assessment reported having acquisition strategies that call for competition post–milestone B at the system or subsystem level.

• About one-third of the major weapon programs in our assessment reported that a configuration steering board meeting has not been held for the program, and few program managers presented descoping options. Under DOD’s revised acquisition policy and in statute, ongoing programs are required to conduct annual configuration steering boards to review requirements changes and significant technical configuration changes that have the potential to result in cost and schedule effects on the program. Since January 2009, 10 programs in our assessment reported changing either a key performance parameter or key system attribute, but only 4 of these reported holding a configuration steering board meeting during that period. In addition, as of June 2010, 12 of the 40 programs in our assessment reported never having held a configuration steering board. In addition to conducting an annual board meeting, the program manager is expected to present descoping options that could reduce program costs or moderate requirements. Only five programs in our assessment reported having presented descoping options and four programs had their options approved.

• The Under Secretary of Defense for Acquisition, Technology and Logistics is beginning to implement a range of efficiency initiatives that focus on affordability, tradeoffs, and portfolio reviews, and are consistent with past GAO recommendations. In September 2010, the Under Secretary issued a memorandum intent on obtaining greater efficiency and productivity in defense spending. Several of the proposed initiatives build on the recent acquisition reforms that DOD has already begun to implement. For example, the memorandum emphasizes the need to set shorter program timelines and manage to them to avoid costly delays in delivering capability to the warfighter. To target affordability, the memo directs program managers to treat affordability as a requirement before programs are started. In

Page 29 GAO-11-233SP Assessments of Selected Weapon Programs

addition, the memo underscores the policy and statutory requirement for making tradeoffs among cost, schedule, and performance objectives using systems engineering analysis prior to system development. To ensure competition throughout a program’s life cycle, the memo proposes requiring the presentation of a competitive strategy at each program milestone. Finally, the Under Secretary proposed conducting portfolio analyses to eliminate redundancies across programs, providing the potential for substantial savings.

Assessments of This section contains assessments of individual weapon programs. Each assessment presents data on the extent to which programs are following a Individual Programs knowledge-based approach to system development and other program information. In total, we present information on 71 weapon programs. For 49 programs, we produced two-page assessments discussing technology, design, and manufacturing knowledge obtained, as well as other program issues. Each two-page assessment also contains a comparison of total acquisition cost from the first full estimate for the program to the current estimate. The first full estimate is generally the cost estimate established at development start; however, for a few programs that did not have such an estimate, we used the estimate at production start instead. For shipbuilding programs, we used their planning estimates if those estimates were available. For programs that began as non–major defense acquisition programs, we used the first full estimate available. Forty-one of these 49 two-page assessments are of major defense acquisition programs, most of which are in development or early production; 3 assessments are of components of major defense acquisition programs, including elements of MDA’s Ballistic Missile Defense System; and 5 assessments are of programs that were projected to become major defense acquisition programs during or soon after our review. In addition, we produced one-page assessments on the current status of 22 programs, which include 13 pre–major defense acquisition programs, 4 major defense acquisition programs that are past their full-rate production decision, 2 elements of MDA’s Ballistic Missile Defense System, 1 major defense acquisition program that was recently terminated, 1 major defense acquisition program that is a commercially derived aircraft, and 1 technology demonstration program.

How to Read the Knowledge For our two-page assessments, we depict the extent of knowledge gained Graphic for Each Program by key points in a program using a stacked bar graph and provide a Assessed narrative summary at the bottom of the first page of each assessment. As

Page 30 GAO-11-233SP Assessments of Selected Weapon Programs

illustrated in figure 8, the knowledge graph is based on three knowledge points. The key indicators for the attainment of knowledge are technology maturity (in orange), design stability (in green), and production maturity (in blue). A “best practice” line is drawn based on the ideal attainment of the three types of knowledge at the three knowledge points. The closer a program’s attained knowledge is to the best practice line, the more likely the weapon will be delivered within estimated cost and schedule. A knowledge deficit at development start—indicated by a gap between the technology maturity attained and the best practice line—means the program proceeded with immature technologies and faces a greater likelihood of cost and schedule increases as risks are discovered and resolved.

Figure 8: Depiction of Notional Weapon System Knowledge as Compared with Knowledge-Based Practices Attainment of Product Knowledge Production,288 design, and technology maturity e

Design and Projection technolo192gy maturity

Desired level of knowledg

Technology maturity 96

0 Development DOD GAO Production start design review decision review Source: GAO.

An interpretation of this notional example would be that system development began with critical technologies that were partially immature, thereby missing knowledge point 1, which is indicated by the orange

Page 31 GAO-11-233SP Assessments of Selected Weapon Programs

diamond. By the design review, technology knowledge had increased, as indicated by the orange bar, but all critical technologies were not yet mature; only 33 percent of the program’s design drawings were releasable to the manufacturer, as indicated by the green bar. Therefore, knowledge point 2, which is indicated by the green diamond, was not attained. At the time of GAO’s review, this program had matured all of its critical technologies and released approximately 75 percent of its design drawings, as indicated by the green bar. When the program plans to make a production decision, it expects to have released all of its design drawings and have half of its critical manufacturing processes in statistical control. The expected knowledge at this future point is captured in the outlined region marked “projection.” This program is not projected to reach knowledge point 3, which is indicated by the blue diamond, by the time it makes a production decision. For shipbuilding programs, knowledge point 1 occurs when a program awards a detailed design and construction contract, and knowledge point 2 occurs when the lead ship starts fabrication. We do not assess production maturity at knowledge point 3 for shipbuilding programs.

Page 32 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: AEHF Advanced Extremely High Frequency (AEHF) Satellite The Air Force’s AEHF satellite system will replenish the existing Milstar system with higher-capacity, survivable, jam-resistant, worldwide, secure communication capabilities for strategic and tactical warfighters. The program includes satellites and a mission control segment. Terminals used to transmit and receive communications are acquired separately by each service. AEHF is an international program that includes Canada, the United Kingdom, and the Netherlands. We assessed the satellite and mission control segments.

Source: © 2009 Lockheed Martin Corporation. All rights reserved.

Concept System development Production

Program Development Design Production First GAO Initial start start review decision launch review capability (4/99) (9/01) (4/04) (6/04) (8/10) (11/10) (TBD)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Lockheed Martin As of Latest Percent Program office: El Segundo, CA 10/2001 12/2009 change Funding needed to complete: Research and development cost $4,843.9 $7,346.3 51.7 R&D: $566.8 million Procurement cost $1,432.6 $5,573.3 289.0 Procurement: $2,765.5 million Total program cost $6,276.5 $12,919.6 105.8 Total funding: $3,332.3 million Program unit cost $1,255.309 $2,153.272 71.5 Procurement quantity: 2 Total quantities 5 6 20.0 Acquisition cycle time (months) 111 170 53.2

The first AEHF satellite (AEHF-1) was launched in Attainment of Product Knowledge August 2010; however, an anomaly with the Production,288 satellite’s propulsion system will delay the satellite design, and technology from reaching its planned orbit and will affect the maturity launch dates of the two satellites currently in e testing. According to the program office, all 14 AEHF critical technologies are mature and its Design and technolo192gy design is stable. However, the program is maturity investigating the cause of the propulsion failure.

We have not assessed the AEHF’s production Desired level of knowledg maturity because the program office does not collect statistical process control data. The Air Technology Force plans to acquire six AEHF satellites that are maturity 96 expected to be clones except for changes to address obsolete parts, and is evaluating requirements and alternatives for meeting future military satellite communication needs beyond the sixth satellite. 0 Development DOD Production GAO start design decision review (9/01) review (6/04) (11/10) (4/04) Page 33 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: AEHF AEHF Program anomaly by the end of 2010. After that, the program office will determine what actions are required to clear AEHF-2 and AEHF-3 for launch. Technology, Design, and Production Maturity According to the program office, all 14 AEHF critical The Air Force plans to procure three additional technologies are mature and its design is stable with AEHF satellites—for a total of six—which will be all of its expected design drawings released. clones of the first three except for obsolete parts. However, the program office is investigating the There will be an approximately 4-year break in cause of a propulsion system failure on the first production between the third and fourth satellites. launched satellite. The propulsion system was not a The program office is working to resolve several critical technology and did not experience problems obsolescence issues and has identified between 12 during prelaunch testing. We did not assess the and 15 flight boxes that have parts that are no longer AEHF’s production maturity because the program available from the original manufacturer. Program office does not collect statistical process control officials do not anticipate encountering significant data. technical challenges, but integrating, testing and requalifying the new parts will require additional Other Program Issues time and money. The notional launch dates for AEHF-1 was launched in August 2010 on an Atlas V satellites four through six are 2017, 2018, and 2020, rocket; however, an anomaly with the propulsion respectively. The Air Force is in the process of system will delay the satellite from reaching its developing a new acquisition program baseline that planned orbit and will affect the launch dates of the includes these satellites. two satellites currently in testing. According to Air Force officials, the satellite separated from the With the cancellation of the Transformational rocket, as planned, and was expected to reach its Satellite Communications System (TSAT) program intended orbit in about 3 months using its liquid in April 2009, the Air Force is in the process of apogee engine and hall current thrusters. The reevaluating its military satellite communications satellite’s software aborted the liquid apogee engine requirements beyond the sixth AEHF satellite. It burns due to low acceleration of the spacecraft, and plans to conduct an analysis of alternatives to assess this engine was rendered unusable because of the options for meeting future requirements, including propulsion system anomaly. The Air Force plans to the possible use of commercial satellite use the propulsion systems designed for controlling communications. and repositioning the satellite to raise the satellite into its planned orbit. As a result, the satellite arrival Program Office Comments on orbit will be delayed by about 7 to 9 months. The AEHF program office provided technical Once the satellite is in its designated orbit, the comments, which were incorporated as appropriate. program office will conduct about 100 days of satellite checkout and system testing before the satellite becomes available for operations. The planned February 2011 launch of AEHF-2 was intended to provide enough time between launches to allow the first satellite to reach orbit and complete on-orbit checkout so any problems identified could be corrected in AEHF-2. Following the same model, the program office will delay the AEHF-2 launch until (1) it is cleared for flight in light of the AEHF-1 propulsion system anomaly and (2) AEHF-1 is on orbit and tested. AEHF-3 will launch about 8 months after AEHF-2. The program’s initial operational capability, which requires two on-orbit satellites to achieve, will be delayed as well. The program office plans to complete a review to identify the root cause of the propulsion system

Page 34 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: AARGM AGM-88E Advanced Anti-Radiation Guided Missile (AARGM) The Navy’s AARGM is an air-to-ground-missile for carrier-based aircraft designed to destroy enemy radio-frequency-enabled surface-to-air defenses. The AARGM is an upgrade to the AGM-88 High Speed Anti-Radiation Missile (HARM). It will utilize the existing HARM propulsion and warhead sections, a modified control section, and a new guidance section with Global Positioning System and improved targeting capabilities. The program is following a phased approach for development. We assessed Phase I.

Source: U.S. Navy.

Concept System development Production

Development Design Low-rate GAO Initial Full-rate start review decision review capability decision (6/03) (3/06) (9/08) (11/10) (TBD) (TBD)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: ATK Missile Systems As of Latest Percent Company 07/2003 07/2010 change Program office: Patuxent River, MD Research and development cost $627.8 $702.8 11.9 Funding needed to complete: Procurement cost $949.4 $1,135.5 19.6 R&D: $7.8 million Total program cost $1,577.2 $1,838.3 16.6 Procurement: $1,018.5 million Program unit cost $.881 $.958 8.7 Total funding: $1,026.3 million Total quantities 1,790 1,919 7.2 Procurement quantity: 1,814 Acquisition cycle time (months) 85 TBD NA

The AARGM program entered production in Attainment of Product Knowledge September 2008 with its critical technologies Production,288 mature and design stable, but without design, and technology demonstrating production maturity. Since then, maturity the program has experienced multiple test delays, e which could affect the program’s planned delivery of initial operational capability in May 2011. The Design and technolo192gy program began operational testing in June 2010 maturity after a 9-month delay due to deficiencies in the missile’s reliability and situational awareness and Desired level of knowledg concerns about the production-representativeness of test missiles. The Navy halted operational Technology testing in September 2010 after hardware and maturity 96 software issues caused a series of missile failures. According to Defense Contract Management Agency officials, the program is working with the contractor to identify the cause of the failures and develop corrective action plans. 0 Development DOD Production GAO start design decision review (6/03) review (9/08) (11/10) (3/06) Page 35 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: AARGM AARGM Program to the operational test failures, the program halted missile deliveries. According to DCMA officials, the program will return all delivered production missiles Technology Maturity and production-representative test rounds to the The AARGM program began system development in contractor to implement any corrective actions. 2003 with its two critical technologies—the Prior to the test failures, the AARGM program office millimeter wave software and radome—nearing had already been working with the contractor to maturity. According to the program office, these improve manufacturing planning, risk identification, technologies were mature and demonstrated in a and reporting. It also included a requirement for the realistic environment when the program entered contractor to develop a detailed manufacturing plan production in September 2008. However, during in the second low-rate production contract, awarded development tests, the program identified in July 2010. deficiencies related to the missile’s reliability and situational awareness that delayed the start of Other Program Issues operational testing. As a result of these deficiencies, The AARGM program has experienced multiple test the program requested and received approval to delays, which could affect the program’s planned defer demonstration of its lethality requirement for a delivery of initial operational capability in May 2011. specific target in a specified scenario until follow-on The program began operational testing in June 2010 operational testing and evaluation. The Director, after a 9-month delay due in part to concerns from Operational Test and Evaluation (DOT&E), noted in DOT&E about the production-representativeness of its fiscal year 2009 assessment that software test missiles. The Navy halted operational testing in development challenges, including those related to September 2010 after hardware and software issues the millimeter wave, continue to pose a risk to the caused a series of missile failures. According to program’s schedule and the missile’s reliability. DOT&E, the program plans to conduct additional developmental tests and a new operational test Design Maturity readiness review in April 2011 before restarting The design of the AARGM appears stable and all operational tests. drawings were releasable to manufacturing by the start of production. Program Office Comments In commenting on a draft of this assessment, the Production Maturity Navy stated that the Program Executive Office, with The AARGM program’s production processes were the support of the Office of the Director of Air not mature when it entered production in September Warfare and the Commander, Operational Test and 2008. According to the program office, the Evaluation Force, decertified AARGM from contractor identified 18 critical manufacturing operational testing as a result of intermittent processes, of which 5 are currently in statistical weapon failures and inaccurate weapon health control. The program plans to demonstrate that all reporting to the aircrew. Upon decertification, the 18 processes are mature during the second lot of program established a review team to assist in root low-rate initial production. Since entering cause analysis and system-level assessments. The production, the program has experienced multiple team found immature manufacturing processes and production delays and operational test failures. software coding errors. A software development and According to Defense Contract Management Agency test program is underway and has demonstrated (DCMA) and DOT&E officials, the test failures were improved performance. Manufacturing processes caused by both hardware and software issues. The are being updated. The first rescreened missiles will hardware failures involved multiple subcontractors enter flight testing by December 2010. An integrated and were primarily attributed to poor parts quality. test phase has been coordinated with DOT&E and DCMA officials said that the contractor is the Commander, Operational Test and Evaluation conducting detailed supplier assessments to Force, to reduce risk upon reentering operational determine the cause of the failures and formulate test. The Italian Air Force remains committed to the corrective action plans. According to DOD’s program. Italian missile deliveries begin in 2011. The manufacturing readiness level deskbook, Navy also provided technical comments, which were assessments of critical suppliers should be incorporated as appropriate. performed before a program enters production. Due

Page 36 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: AB3 Apache Block III (AB3) The Army’s Apache Block III (AB3) program will upgrade AH-64D Longbow Apache helicopters. It is expected to improve performance, situational awareness, lethality, survivability, interoperability, and the prevention of friendly fire incidents. Each Apache goes to the factory for hardware changes. Software improvements can be installed in the field, which reduces the time an aircraft is away from the unit and increases the training time for soldiers in the field. Upgraded AH-64Ds are scheduled to enter service starting in 2011.

Source: U.S. Army.

Concept System development Production

Development System Production Low-rate GAO Full-rate Initial start design review design review decision review production capability (7/06) (1/08) (3/09) (10/10) (11/10) (7/12) (5/13)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Boeing As of Latest Percent Program office: Huntsville, AL 08/2006 07/2010 change Funding needed to complete: Research and development cost $1,138.4 $1,629.6 43.1 R&D: $799.7 million Procurement cost $5,996.7 $11,112.6 85.3 Procurement: $8,726.3 million Total program cost $7,135.1 $12,742.2 78.6 Total funding: $9,526.0 million Program unit cost $11.852 $18.334 54.7 Procurement quantity: 682 Total quantities 602 695 15.4 Acquisition cycle time (months) 79 82 3.8 Latest cost and quantities include 56 new-build helicopters.

In October 2010, the AB3 received approval to Attainment of Product Knowledge enter production with mature critical technologies Production,288 and a stable hardware design. AB3 upgrades design, and technology involve a time-phased series of hardware and maturity software-related technical insertions. According e to program officials, the design reviews for the hardware portion of the program have been held, Design and technolo192gy all the expected design drawings are releaseable maturity to manufacturing, and the last two software- related design reviews are scheduled for fiscal Desired level of knowledg years 2012 and 2014. In May 2010, the Director, Defense Research and Engineering, assessed the Technology AB3 as ready for production using engineering maturity 96 manufacturing readiness levels, a metric that includes technology and design maturity and production readiness. The AB3 program experienced a Nunn-McCurdy unit cost breach of the critical threshold in June 2010, due to the 0 Development DOD Production GAO addition of 56 new-build helicopters to the start design decision review upgrade program. (7/06) review (10/10) (11/10) (3/09) Page 37 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: AB3 AB3 Program involved taking legacy aircraft and remanufacturing them with upgraded capabilities. However, decreases in the numbers of legacy aircraft available Technology Maturity for remanufacture due to combat losses, combined The AB3 program’s one critical technology—an with increasing wartime demands and the addition improved drive system—is mature. This is the first of a 13th Combat Aviation Brigade, has resulted in a time this technology will be used in a helicopter new acquisition strategy that includes acquiring 56 transmission, and it is expected to provide more new-build aircraft. Since new-build Apaches cost power and be more reliable than the existing three times more than a remanufactured Apache, a transmission. A developmental test aircraft has cost breach occurred. As part of its Nunn-McCurdy successfully completed flight tests and has restructuring, the AB3 will be divided into two demonstrated the maturity of the drive system in a separate major defense acquisition programs—one realistic environment. for remanufactured aircraft and one for new builds. This division will permit visibility into the cost, Design Maturity schedule, and performance of both programs. Even The AB3 hardware design is stable. AB3 upgrades after the program restructuring, risks remain in the involve a time-phased series of hardware and AB3 program. An analysis by DOD’s Program software-related technical insertions. The design Assessment and Root Cause Analyses office noted reviews for the hardware portion of the program that even though the AB3 contractor has performed have been held and all the expected design drawings well on its development contract, increasing are releasable to manufacturing. According to software content, extensions of the development program officials, the last two critical design schedule, and the ability of the contractor to provide reviews, which are software-related, should not production aircraft at prices consistent with the significantly affect the total number of design existing program baseline, all pose risks for the drawings. These reviews are scheduled for fiscal current program. years 2012 and 2014—after upgraded AH-64Ds are scheduled to start to enter service. According to Program Office Comments program officials, the AB3 program uses a contract In commenting on a draft of this assessment, the provision requiring the completion of 85 to 90 Army provided technical comments, which were percent of the estimated design drawings for each incorporated where appropriate. design review as a mechanism for ensuring design stability. In addition, the success of each design review determines whether the program will move forward.

Production Maturity In October 2010, AB3 received approval to enter production. We did not assess production maturity because the program has not started to collect statistical process control data. However, the Director, Defense Research and Engineering, assessed the AB3 as ready for production in May 2010. The assessment used engineering manufacturing readiness levels, a metric that includes technology and design maturity and production readiness, as the basis for reaching this conclusion.

Other Program Issues The AB3 program experienced a Nunn-McCurdy unit cost breach of the critical threshold in June 2010 due to the addition of new-build helicopters to the upgrade program. The original AB3 program

Page 38 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Army IAMD Army Integrated Air and Missile Defense (Army IAMD) The Army’s Integrated Air and Missile Defense (IAMD) program is being developed to network sensors and weapons and a common battle command system across a single integrated fire control network to support the engagement of air and missile threats. The IAMD Battle Command System (IBCS) will provide control and management for IAMD sensors and weapons, such as the Joint Land-Attack Cruise Missile Defense Elevated Netted Sensor System and PATRIOT through an interface module that supplies battle management data and enables networked operations.

Source: Northrop Grumman.

Concept System development Production

Technology Development GAO Design Low-rate Initial Full-rate development start review review decision capability decision (2/06) (12/09) (11/10) (8/11) (12/14) (8/16) (5/17)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Northrop Grumman As of Latest Percent Space & Mission Systems Corp 12/2009 12/2009 change Program office: Huntsville, AL Research and development cost $1,571.4 $1,571.4 0.0 Funding needed to complete: Procurement cost $3,382.6 $3,382.6 0.0 R&D: $1,177.4 million Total program cost $4,954.0 $4,954.0 0.0 Procurement: $3,382.6 million Program unit cost $16.737 $16.737 0.0 Total funding: $4,560.0 million Total quantities 296 296 0.0 Procurement quantity: 285 Acquisition cycle time (months) 80 80 0.0

IAMD entered development in December 2009 Attainment of Product Knowledge with its four critical technologies nearing maturity. Production,288 However, according to program officials, the design, and technology technologies will not be fully mature until after the maturity design review in August 2011. As a result, the e program will not have demonstrated that the proposed design meets requirements until after Design and technolo192gy the design review, which puts it at risk for late maturity design changes. The platform the IBCS was originally planned to be fielded on will not be Desired level of knowledg available when production begins in 2014, but according to program officials, an alternative has Technology been selected. The cost and schedule of the IAMD maturity 96 program may also be affected by an Army proposal to substitute the IBCS for the current battle management system under development for the Medium Extended Air Defense System Not Not assessed assessed (MEADS). If this option is selected for any of the 0 Development GAO DOD Production MEADS partners, the development schedule for start review design decision IAMD will need to be synchronized with MEADS. (12/09) (11/10) review (12/14) (8/11) Page 39 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Army IAMD Army IAMD Program program. According to Army officials, a number of issues would need to be resolved before the proposal could be adopted. These issues include a Technology Maturity decision between the IBCS and BMC4I contractors IAMD entered development in December 2009 with about whether the substitution is feasible and a its four critical technologies—integrated battle decision about whether to adopt it for any or all of command, integrated defense design, integrated fire the MEADS partners. If any of the MEADS partners control network, and distributed track agree to the substitution, the development schedule management—nearing maturity. In August 2009, the for IAMD will need to be synchronized with MEADS. Office of the Deputy Assistant Secretary of the Army Specifically, IBCS would be needed for integration for Research and Technology approved a technology testing prior to the MEADS low-rate initial readiness assessment that stated that all of the production decision, currently planned for critical technologies were tested in a relevant November 2012. environment using digital simulations. In addition, the integrated fire control network and the Program Office Comments distributed track management technologies were In commenting on a draft of this assessment, the demonstrated through hardware tests, and the Army stated that the IAMD program entered the integrated defense design was demonstrated engineering and manufacturing development (EMD) through a prototype. Program officials estimate that phase in December 2009 after a competitive the technologies will be mature by the IAMD prototyping phase lasting 15 months. During this production decision in 2014, but not by its planned phase, the competitors (Raytheon and Northrop) August 2011 design review. developed IBCS prototypes which were demonstrated to the government prior to the Design Maturity selection of one contractor (Northrop). Both The IAMD program is preparing for its August 2011 contractors were assessed at technology readiness design review; however, the risk of late and levels necessary for entry into the EMD phase. potentially costly design changes will persist Subsequently, Northrop’s design was reassessed in because the program will not have demonstrated December 2010, and all critical technologies were at that the program’s critical technologies are fully the level needed for the current phase of the mature or the proposed design meets requirements program. The program is on track to conduct the by then. The Army completed a partial preliminary critical design review in August 2011. With regards design review prior to development start for the to the insertion of the IBCS into the MEADS IAMD and its components. The IBCS preliminary program, the Army and the Office of the Secretary of design review is complete, and the reviews for Defense agreed to withdraw the proposal based on IAMD, the interface modules, and the overall cost and schedule considerations. Any use of the integration of the components were expected to be IBCS with the MEADS components will be after complete in November 2010. Officials stated that the fiscal year 2016. The Army also provided technical Army expects to modify the IBCS design because the comments, which were incorporated as appropriate. platform it was planned to be fielded on—the High Mobility Multipurpose Wheeled Vehicle—will not be available when it enters production in 2014. The Army and the contractor evaluated alternatives and selected a chassis from the Family of Medium Tactical Vehicles (FMTV) as a replacement. The program is now working on integrating the FMTV chassis into the design for the IBCS.

Other Program Issues The cost and schedule of the IAMD program could be affected by an Army proposal to substitute the IBCS for the Battle Management Command, Control, Communications, Computer, and Intelligence (BMC4I) system under development for the MEADS

Page 40 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: B-2 EHF SATCOM Increment 1 B-2 Extremely High Frequency (EHF) SATCOM Capability, Increment 1 The Air Force’s B-2 EHF satellite communications upgrade is being developed in three increments. Increment 1 upgrades the computing system speed and storage capacity of the current avionics infrastructure to facilitate future B-2 upgrades. Increment 2 will provide connectivity through low- observable antennas and radomes, and includes nonintegrated Family of Advanced Beyond Line-of- Sight Terminals and related hardware. Increment 3 will enable connectivity with the Global Information Grid and net-ready capability. We assessed Increment 1.

Source: U.S. Air Force.

Concept System development Production

Program Development Design GAO Low-rate Full-rate Required Last start start review review decision decision assets available procurement (3/02) (2/07) (10/08) (11/10) (7/11) (7/12) (3/14) (2014)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Northrop Grumman As of Latest Percent Program office: Wright-Patterson AFB, 05/2007 07/2010 change OH Research and development cost $577.7 $501.9 -13.1 Funding needed to complete: Procurement cost $121.7 $116.7 -4.1 R&D: $119.7 million Total program cost $699.4 $618.6 -11.6 Procurement: $116.7 million Program unit cost $33.303 $30.929 -7.1 Total funding: $236.4 million Total quantities 21 20 -4.8 Procurement quantity: 16 Acquisition cycle time (months) 85 85 0.0

According to the program office, the B-2 EHF Attainment of Product Knowledge SATCOM Increment 1 will have mature critical Production,288 technologies and a stable design by its planned design, and technology July 2011 low-rate initial production decision. The maturity program office also plans to demonstrate critical e manufacturing processes using a pilot production line and high levels of manufacturing readiness for Design and technolo192gy two key technologies prior to the production maturity decision. The program expects an operational

assessment to be completed by the Air Force Desired level of knowledg Projection Operational Test and Evaluation Center in early 2011 to support the production decision. The B-2 Technology EHF SATCOM Increment 1 program completed maturity 96 software certification in April 2010 and flight testing began in September 2010, after a 5-month delay. According to the program office, this delay has added pressure to the test schedule and the program’s plan to begin initial operational test and 0 Development DOD GAO Production evaluation in fiscal year 2012. start design review decision (2/07) review (11/10) (7/11) (10/08) Page 41 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: B-2 EHF SATCOM Increment 1 B-2 EHF SATCOM Increment 1 recovering from early software development issues Program that delayed the start of developmental test and evaluation by 9 months. Flight testing began in September 2010 after a 5-month delay. The delay has Technology and Design Maturity added pressure to the current test schedule and The B-2 EHF SATCOM Increment 1 program entered increased the schedule risk for the start of initial system development in February 2007 with all six of operational test and evaluation in fiscal year 2012. its critical technologies nearing maturity. The According to the program office, the delay was a program office expects all critical technologies to be result of installation issues, test aircraft concerns, mature and flight-qualified by the program’s planned and higher B-2 testing priorities. The program office July 2011 production decision. The B-2 EHF has taken steps to address the installation issues and Increment 1 design also appears stable. According to the health of the test aircraft. the program office, all of the expected drawings were releasable at the October 2008 design review Program Office Comments and the number of design drawings has not grown. In commenting on a draft of this assessment, the B-2 program office noted that despite the delays in The development and successful integration of new software development and in the start of flight test, disk drive units and integrated processing units is a the B-2 EHF Increment 1 schedule remains a year primary objective for Increment 1. The Air Force has ahead of the July 2012 threshold date for its low-rate completed disk drive unit qualification and design initial production decision, and 7 months ahead of verification without discovering any significant the September 2012 threshold date for completion of issues, and integrated processing unit durability and initial operational test and evaluation. The Air Force airworthiness testing was also completed. also provided technical comments, which were incorporated as appropriate. Production Maturity Although we did not fully assess production maturity because the program does not have statistical process control data, we did assess aspects of production maturity. According to the program office, the B-2 EHF SATCOM Increment 1 is based on commercial-off-the-shelf technology with proven manufacturing processes. The program is implementing other practices that will also help demonstrate production maturity. For example, according to the program office, system-level development testing of a fully configured, production-representative prototype in its intended environment began in July 2010, and critical manufacturing processes will be demonstrated on a pilot production line using production- representative articles prior to the July 2011 production decision. In addition, the program plans to complete a manufacturing readiness level assessment to support a production readiness review in April 2011 and demonstrate a high level of manufacturing readiness for the disk drive units and integrated processing units.

Other Program Issues B-2 EHF SATCOM Increment 1 testing is progressing; however, earlier delays have added schedule risk to the test program. The program completed software certification in April 2010, after

Page 42 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: BMDS: GMD BMDS: Ground-Based Midcourse Defense (GMD) MDA’s GMD is being fielded to defend against limited long-range ballistic missile attacks during their midcourse phase. GMD consists of an interceptor with a 3-stage booster and kill vehicle, and a fire control system that formulates battle plans and directs components integrated with BMDS radars. We assessed the maturity of all GMD critical technologies, as well as the design of the Capability Enhancement II (CE-II) configuration of the Exoatmospheric Kill Vehicle (EKV), which began emplacements in fiscal year 2009.

Source: Missile Defense Agency.

Technology/system development Initial capability

Program Directive to field 1st emplaced Initial 1st CE-I 1st emplaced Failed CE-II GAO start initial capability CE-I capability successful CE-II intercept test review interceptor intercept interceptor (2/96) (12/02) (7/04) (10/04) (9/06) (10/08) (1/10) (11/10)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Boeing Latest Percent Program office: Redstone Arsenal, AL As of 02/2010 change Funding FY11-FY15: Research and development cost NA $37,844.0 NA R&D: $5,512.2 million Procurement cost NA $0.0 NA Procurement: $0.0 million Total program cost NA $38,082.4 NA Total funding: $5,553.6 million Program unit cost NA NA NA Procurement quantity: 0 Total quantities NA NA NA Acquisition cycle time (months) NA NA NA Columns include costs from program inception through fiscal year 2015. Totals do not include the future cost of the European component.

MDA continues to put the GMD program at risk Attainment of Product Knowledge for further cost growth and delays as it buys and Production,288 emplaces CE-II interceptors before all the critical design, and technology technologies have been demonstrated in a realistic maturity environment. After a 1-year delay, MDA tested the e CE-II EKV in January 2010, but it failed to achieve an intercept. GMD reconducted the test in Design and technolo192gy December 2010 and although the booster and EKV maturity were successfully launched, it again failed to achieve an intercept. Almost all of the CE-II kill Desired level of knowledg vehicles currently under contract will have been delivered before the test is successfully Technology conducted. Moreover, developmental testing is maturity 96 expected to continue until fiscal year 2021, well after the last planned EKV deliveries. Due to the concurrent testing and production of the CE-II EKV, the program could experience costly late Not assessed design changes and retrofits if problems are 0 Development DOD GAO Production discovered during flight testing. start design review decision (NA) review (11/10) (TBD) (5/06) Page 43 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: BMDS: GMD BMDS: GMD Program program is at risk for costly late design changes and retrofits if problems are discovered during flight testing. Technology Maturity All nine technologies in the GMD operational In fiscal year 2010, GMD conducted a nonintercept configuration are mature, but two technologies flight test of its two stage GBI, which was originally developed for the CE-II interceptor—an upgraded designed for a European site. Although all flight test infrared seeker and onboard discrimination—are objectives were achieved, an EKV anomaly was nearing maturity. Although the program delivered experienced that might affect system performance. and fielded more upgraded interceptors in fiscal year 2010, its full capability has yet to be verified MDA is currently developing plans to sustain the through flight tests. GMD element through 2032; however, key unknowns and a lack of analysis have hindered these efforts. In Design Maturity fiscal year 2010, GMD continued to develop its fleet The design of the enhanced interceptor appears rotation strategy and aging and surveillance test plan stable with all of its expected drawings released to and completed its stockpile reliability plan. GAO has manufacturing. However, the design could still been unable to fully assess these efforts because change because two technologies are still being they lack key analysis. For example, the sufficiency developed and have not had their capability verified of the planned inventory of operational GBIs is through flight testing. based on various assumptions, including the reliability of the interceptor. However, Production Maturity developmental testing is expected to continue until We did not assess the maturity of the production at least 2021 and the reliability of the interceptor is processes for the GMD interceptors. The program is not fully known. buying interceptors for operational use, but officials do not plan to make an official production decision Program Office Comments or collect statistical control data because the The program office provided technical comments, planned quantities are small. However, according to which were incorporated as appropriate. GMD officials, the program does track defects per unit for each major interceptor component. In addition, GMD employs a manufacturing capability assessment process in which all critical manufacturing indicators are assessed on a monthly basis.

Other Program Issues GMD continues to concurrently develop, manufacture, and field the CE-II EKVs putting it at risk for further cost growth, schedule delays, and performance shortfalls in delivered capability. After experiencing over a 1-year delay, GMD conducted an intercept flight test to assess the capability of the CE-II EKV in January 2010; however, it did not intercept the target because of a failure in the EKV. GMD reconducted this test in December 2010. Although the booster was successfully launched and deployed the EKV, it failed to intercept the target. The next flight test will be determined after identification of the cause of the failure. Although the emplaced CE-II ground-based interceptors (GBI) have not been declared operational, the production of the CE-II is nearly complete. Consequently, the

Page 44 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: BMDS: THAAD BMDS: Terminal High Altitude Area Defense (THAAD) MDA’s THAAD is being developed as a rapidly- deployable, ground-based missile defense system with the capability to defend against short- and medium-range ballistic missiles during their late midcourse and terminal phases. A THAAD battery includes interceptor missiles, three to six launchers, an X-band radar, and a fire control and communications system. MDA is scheduled to deliver the first two of nine planned THAAD batteries to the Army in fiscal years 2011 and 2012 for initial operational use.

Source: Missile Defense Agency.

Technology/system development Initial capability

Program Transition Contract award Contract award GAO Materiel Battery 1 Battery 2 start to MDA for Batteries for Battery 3 review release initial capability initial capability 1 and 2 interceptors to Army delivery delivery (1/92) (10/01) (12/06) (9/10) (11/10) (3/11) (9/11) (3/12)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Lockheed Martin Latest Percent Program office: Huntsville, AL As of 02/2010 change Funding FY11-FY15: Research and development cost NA $15,973.3 NA R&D: $1,892.4 million Procurement cost NA $4,412.3 NA Procurement: $3,881.9 million Total program cost NA $21,156.2 NA Total funding: $6,544.9 million Program unit cost NA NA NA Procurement quantity: NA Total quantities NA NA NA Acquisition cycle time (months) NA NA NA Columns include costs from the program’s inception through fiscal year 2015.

THAAD has mature technologies and has had a Attainment of Product Knowledge production contract since 2006, but the program is Production,288 still experiencing design and production issues. design, and technology Problems with a safety switch have caused maturity interceptor production issues, design changes, e and schedule delays. Deliveries of Batteries 1 and 2 have been delayed at least 1 year; the number of Design and technolo192gy design drawings has increased by more than 20 maturity percent since production began due to the switch

redesign and other related changes; and the Desired level of knowledg Army’s acceptance of THAAD batteries has been delayed 6 months until testing on the switch is Technology complete. Most THAAD ground component maturity 96 deliveries for Batteries 1 and 2 are complete, but there will be a production gap of more than a year for future battery ground and missile components, which could increase costs. In fiscal year 2010, Not assessed THAAD successfully proved out its objective 0 Development DOD Production GAO software in flight testing. start design start review (6/00) review (12/06) (11/10) (12/03) Page 45 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: BMDS: THAAD BMDS: THAAD Program launchers are expected to complete the government acceptance process by the end of the third quarter of fiscal year 2011. Technology and Design Maturity The THAAD program’s major components—the fire Other Program Issues control and communications system, interceptor, The THAAD program delayed its conditional release launcher, and radar—are mature. According to the of batteries to the Army from September 2010 until program office, the prime contractor has released March 2011 because of ongoing safety issues with 100 percent of the expected design drawings. interceptor components. For the release to occur, However, the number of design drawings has the Army must certify that the batteries are safe, increased by more than 20 percent since production suitable, and logistically supported. According to started in December 2006, due primarily to fire program officials, the results from optical block control risk reduction efforts, design changes for a safety testing in February 2011 are needed before safety system called an optical block that prevents the release board can make its decision. inadvertent launches, and associated changes to the flight sequencing assembly, which houses the optical The THAAD program is facing a production gap that block. Additional drawings or design work could poses cost and schedule risks. Product qualification still be required based on the results of remaining issues delayed contract award for Battery 3 ground and flight testing. interceptors by approximately 6 months to the end of fiscal year 2010, and there will be as much as a 1- Production Maturity year production gap for some interceptor MDA awarded a contract for its first two initial components. The program did not plan to award the operational batteries in December 2006 before contract for THAAD ground components and completing developmental testing on all the system’s Battery 4 interceptors until the first quarter of fiscal critical components and has experienced production year 2011, which means there will be a production delays as a result. While we did not assess THAAD’s gap of up to 3 years for some ground components. overall production maturity because the program As a result of these gaps, the program will have to has not collected statistical process control data on retrain workers and recertify and requalify parts. its critical manufacturing processes, the program’s The effect of these gaps on cost is not yet known. production readiness assessments highlighted a number of risks including design producibility and Despite test delays due to target issues, the program qualification of critical components. The delivery of was able to conduct one flight test in June 2010 to the first two batteries has been delayed by at least a successfully demonstrate the complete objective year after these risks materialized in parts of the software for the THAAD battery. THAAD interceptor. A qualified optical block design failed during integration qualification testing in early Program Office Comments fiscal year 2010 due to contamination during Program officials stated that the THAAD program is manufacturing. The program changed to cleaner significantly more mature than indicated in the manufacturing processes and subsequently “Attainment of Product Knowledge” graph and completed this qualification testing in September associated language. At production start all 2010. According to program officials, the program production design was completed except for two plans to develop a more producible design of the items associated with the fire control and optical block for use on future THAAD battery interceptor. At the time of this review, none of the interceptors. All of the ground components interceptors for the THAAD batteries were necessary for Batteries 1 and 2 have been delivered delivered, but all THAAD system and component except for the launchers which are experiencing a 2 qualifications were completed except for two year delay in completing the government acceptance interceptor-related tests. All but one of the process because of production issues as well as subassemblies for each of the 50 interceptors was delays to the qualification process due to design delivered. Other technical comments were changes. In addition, discoveries during a recent incorporated as appropriate. ground test have led to further design changes. The

Page 46 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: BAMS UAS Broad Area Maritime Surveillance (BAMS) Unmanned Aircraft System (UAS) The Navy’s Broad Area Maritime Surveillance Unmanned Aircraft System (BAMS UAS) is intended to provide a persistent maritime intelligence, surveillance, and reconnaissance (ISR) capability. BAMS UAS will be part of a family of maritime patrol and reconnaissance systems that recapitalizes the Navy’s airborne ISR assets. Increments 2 and 3 of the program will upgrade the system’s communication relay and add a signals intelligence capability. We assessed Increment 1.

Source: U.S. Navy.

Concept System development Production

Program/ GAO Design Low-rate Initial Full development start review review decision capability capability (4/08) (11/10) (1/11) (5/13) (12/15) (2019)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Northrop Grumman As of Latest Percent Systems Corporation 02/2009 12/2009 change Program office: Patuxent River, MD Research and development cost $3,094.5 $3,150.0 1.8 Funding needed to complete: Procurement cost $9,185.3 $9,501.8 3.4 R&D: $2,106.5 million Total program cost $12,656.7 $13,031.5 3.0 Procurement: $9,501.8 million Program unit cost $180.811 $186.165 3.0 Total funding: $11,988.1 million Total quantities 70 70 0.0 Procurement quantity: 65 Acquisition cycle time (months) 92 92 0.0

The BAMS UAS program began development in Attainment of Product Knowledge 2008 with all technologies nearing maturity and Production,288 plans to demonstrate its design is stable by its design, and technology critical design review in January 2011. The maturity program office continues to monitor six watch-list e items that were identified in a 2007 independent technology readiness assessment that could affect Design and technolo192gy the program’s cost, schedule, and performance. maturity The BAMS UAS program plans to enter production in May 2013. According to program officials, the Desired level of knowledg BAMS air vehicle is based on the RQ-4B Global Hawk and uses sensor components or entire Technology subsystems from other existing platforms. There maturity 96 are some structural changes to the airframe, but none of these require significant changes to manufacturing processes. The program expects to purchase two developmental aircraft and begin Not assessed testing prior to production. 0 Development GAO DOD Production start review design decision (4/08) (11/10) review (5/13) (1/11) Page 47 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: BAMS UAS BAMS UAS Program hub, and basing options for both UASs. According to program staff, thus far, only minor changes to the configuration of the BAMS UAS are anticipated. Technology Maturity DOD and the Navy certified that all BAMS UAS The BAMS UAS program is also continuing to technologies were nearing maturity and had been leverage knowledge from the BAMS demonstrator demonstrated in a relevant environment before the program. The demonstrator consists of two Block 10 start of system development. The program is Global Hawk UASs. It is used to support BAMS UAS monitoring six watch-list items that were identified design activity, test ground station capabilities, and in a 2007 independent technology readiness develop concepts of operations. For example, a assessment, which could cause cost, schedule, or BAMS UAS official noted the demonstrator has been performance issues during development. According successful using mission operating bases, which to the program, the Navy conducted an additional conduct data analysis and operate flight controls, independent technology readiness assessment after located in the continental United States. As a result, the program’s February 2010 preliminary design the program office plans to update its concept of review. DOD was reviewing the results at the time of operations to reflect this lesson learned. According our assessment. to the program, forward operating bases, responsible for launch and recovery of the aircraft, Design Maturity will remain in theater as planned. The BAMS UAS program expects the air vehicle’s design to be stable by its January 2011 critical design The BAMS UAS program poses a significant review. However, the program will be at risk for software development challenge. The program will design changes until it integrates all of its key utilize more than 6 million lines of code, including subsystems and components and tests them through more than 1 million lines of new code. Total lines of an integration laboratory or an early system code have increased by about 13 percent since prototype demonstration. This will not occur until development start, which according to the program, January 2012. According to the program office, were the result of selecting a different sense-and- about 79 percent of the BAMS UAS air vehicle’s avoid radar subsystem, and shifting from reused expected drawings are releasable to manufacturing code to new software for the synthetic aperture and 8 of the 11 subsystem critical design reviews radar. The program is closely monitoring the have been successfully completed. software effort and software is being developed in three blocks of capability to decrease risks. Production Maturity The BAMS aircraft is based on the Global Hawk  Program Office Comments RQ-4B currently in production, and, according to According to the Navy, the program continues to program officials, uses sensor components or entire meet its cost, schedule, and performance subsystems from other existing platforms. There are requirements. In support of the engineering and some significant changes to the airframe, such as manufacturing development decision, the Navy deicing and structural reinforcements for the wings, stated that the Office of the Secretary of Defense but none of these require significant manufacturing assessed the technology and determined BAMS had process changes. The program expects to purchase been demonstrated in a relevant environment, since two developmental aircraft and begin testing them the program leverages existing DOD investment in prior to production. its airframe, engine, avionics, payloads, and software. The Navy also stated that the program is Other Program Issues capturing and applying lessons learned from the In 2010, the Joint Requirements Oversight Council programs it is leveraging in order to maximize its directed the Navy and Air Force to seek efficiencies effectiveness and efficiency. Furthermore, the Navy in the Global Hawk and BAMS UAS programs. stated that the BAMS UAS and Air Force Global According to BAMS UAS program officials, the Air Hawk programs continue to work closely together to Force and Navy programs are investigating seek synergistic opportunities in all phases of the commonality opportunities in areas such as sense- programs. The Navy also provided technical and-avoid capabilities, a common ground control comments, which were incorporated as appropriate. station architecture, a consolidated maintenance

Page 48 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: C-5 RERP C-5 Reliability Enhancement and Reengining Program (C-5 RERP) The Air Force’s C-5 RERP is one of two major upgrades for the C-5. The RERP is designed to enhance the reliability, maintainability, and availability of the C-5 by replacing the propulsion system; modifying the mechanical, hydraulic, avionics, fuel, and landing gear systems; and making required structural modifications. Together with the C-5 Avionics Modernization Program, these upgrades are intended to improve C-5 mission capability rates and reduce total ownership costs.

Source: Lockheed Martin.

Concept System development Production

Program Development Design Low-rate Full-rate GAO Last start start review decision decision review procurement B-model (2/00) (11/01) (4/04) (3/08) (10/10) (11/10) (2014)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Lockheed Martin As of Latest Percent Program office: Wright-Patterson AFB, 11/2001 03/2010 change OH Research and development cost $1,726.6 $1,760.6 2.0 Funding needed to complete: Procurement cost $9,013.3 $5,582.9 -38.1 R&D: $49.0 million Total program cost $10,743.7 $7,348.8 -31.6 Procurement: $4,409.2 million Program unit cost $85.267 $141.323 65.7 Total funding: $4,458.2 million Total quantities 126 52 -58.7 Procurement quantity: 40 Acquisition cycle time (months) 100 135 35.0

The C-5 RERP entered production in March 2008 Attainment of Product Knowledge with mature technologies and a design that was Production,288 nearing completion. We did not assess production design, and technology maturity because the program office does not maturity require process control data to be collected. To e determine the program’s readiness to enter full- rate production, the program completed a Design and technolo192gy manufacturing readiness assessment, which maturity concluded its manufacturing processes were capable, in control, and affordable. The Desired level of knowledg assessment was performed on the first low-rate initial production aircraft and may not reflect all Technology manufacturing risks. A production aircraft was maturity 96 not used for initial operational testing. The Air Force test organization found the system to be effective, suitable, and mission capable; however, it noted that incomplete development and testing

of the aircraft’s defensive systems and thrust 0 Development DOD Production GAO reversers increased the risk of operating in certain start design decision review environments. (11/01) review (3/08) (11/10) (4/04) Page 49 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: C-5 RERP

C-5 RERP Program Other Program Issues The Commander, Air Force Operational Test and Evaluation Center, found the C-5 RERP to be Technology and Design Maturity effective, suitable, and mission-capable during According to an independent technology readiness operational testing conducted from October 2009 to assessment conducted in October 2001, the C-5 January 2010. The Air Force did not provide a low- RERP’s technologies are mature. In addition, the C-5 rate initial production aircraft for operational testing RERP design is stable. as recommended by the Director, Operational Test and Evaluation, because one was not available. Production Maturity However, according to program officials, a The C-5 RERP program entered production in March production-representative aircraft was used in 2008. We did not assess production maturity because operational testing. According to the test report, the the program office does not require process control modified development aircraft increased the C-5 data to be collected as part of the production maximum operating weight, significantly surpassed contract. In order to determine the program’s its reliability requirement, and performed its readiness to enter full-rate production, the Air Force required mission better than the C-5 legacy fleet. and the prime contractor, Lockheed Martin, However, the Air Force test organization also noted performed a manufacturing readiness assessment in that incomplete development and testing of the early 2010. According to the final report, a aircraft’s defensive systems and thrust reversers manufacturing readiness assessment is normally increased the risk of operating in certain conducted near the end of low-rate initial environments. production. However, the assessment was performed on the first low-rate initial production In February 2010, DOD released the Mobility aircraft while it was in production and may not Capabilities Requirements Study–2016, which reflect all the manufacturing risks. Program officials concluded that the Air Force has excess strategic stated that conducting the assessment at the airlift capacity. As a result, the Air Force is beginning of low-rate initial production was driven requesting approval to retire 22 C-5A aircraft. This by full-rate production decision requirements would reduce the number of aircraft in the C-5 established in an acquisition decision memorandum Avionics Modernization Program, but it would not and the expected date of the decision. The Air Force affect the number of aircraft in the C-5 RERP accepted delivery of the first production aircraft in program. October 2010. Program Office Comments The C-5 RERP program’s manufacturing assessment The Air Force provided technical comments to a concluded that the manufacturing risk was draft of this assessment, which were incorporated as understood and that the manufacturing processes appropriate. for the system were capable, in control, and affordable. The assessment identified what program officials believe to be two minor issues with the aircraft engine’s thrust reverser, which will not affect production, and an air exit door that could affect production. According to program officials, design changes are being made to the thrust reverser to prevent freezing and will be ready for testing in April 2011. The thrust reverser modifications will be installed on all modernized C-5 aircraft, including those that have already been upgraded. The malfunctioning air exit door will be addressed through a change to production processes and will not require additional flight testing. The program expects all aircraft in the modernized C-5 fleet to eventually receive modified air exit doors.

Page 50 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: C-130 AMP C-130 Avionics Modernization Program (C-130 AMP) The Air Force’s C-130 AMP will standardize the cockpit and avionics for three combat configurations of the C-130 fleet. The program is intended to ensure the C-130 global access and deployability by satisfying navigation and safety requirements, installing upgrades to the cockpit systems, and replacing many systems no longer supportable due to diminishing manufacturing resources. It is also expected to increase the reliability, maintainability, and sustainability of the upgraded aircraft.

Source: C-130 Avionics Modernization Program.

Concept System development Production

Development Design Low-rate GAO Full-rate Last start review decision review decision procurement (7/01) (8/05) (6/10) (11/10) (2/13) (2018)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Boeing As of Latest Percent Program office: Wright-Patterson AFB, 07/2001 04/2010 change OH Research and development cost $764.0 $1,895.6 148.1 Funding needed to complete: Procurement cost $3,307.2 $4,100.0 24.0 R&D: $55.9 million Total program cost $4,071.2 $5,995.6 47.3 Procurement: $3,942.8 million Program unit cost $7.844 $27.129 245.8 Total funding: $3,998.7 million Total quantities 519 221 -57.4 Procurement quantity: 214 Acquisition cycle time (months) NA NA NA

The C-130 AMP program entered production in Attainment of Product Knowledge June 2010 with mature technologies and a stable Production,288 design. The program reported that it design, and technology demonstrated critical manufacturing processes maturity prior to production; however, it did not assess e manufacturing readiness levels at key suppliers and installation facilities. The first two aircraft to Design and technolo192gy receive the AMP upgrade have begun the maturity modification process. During low-rate production,

the program plans to qualify a second contractor Desired level of knowledg to compete for the full-rate production contract. In February 2010 the program reported a Nunn- Technology McCurdy unit cost breach of the significant cost maturity 96 growth threshold, which it attributed to factors such as the omission of training devices and adequate spares from initial estimates, and delays in the production decision. The program has been restructured and planned dates for key events 0 Development DOD Production GAO have been pushed back by more than 1 year. start design decision review (7/01) review (6/10) (11/10) (8/05) Page 51 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: C-130 AMP C-130 AMP Program planned contract award for the second source have been delayed by about 4 months to early 2011 and early 2012, respectively, because the program has Technology and Design Maturity changed the acquisition approach for this effort. The C-130 AMP program’s three current critical These delays could affect the program’s planned technologies—global air traffic management, entry into full-rate production in early 2013. defensive systems, and combat delivery navigator removal—are mature. After a Nunn-McCurdy unit The C-130 AMP program has completed cost breach of the critical cost growth threshold in developmental testing, which identified key 2007, the program was restructured and the number deficiencies in the areas of crew workload and of critical technologies was cut from six to three. avionics. Specifically, testing found that the core The removed technologies were intended for Special avionics were not well adapted to the C-130 tactical Mission C-130 aircraft configurations, which were missions, and identified the need to reduce crew eliminated from the program. The design of the  workload during airdrop, low-level, and certain C-130 AMP combat delivery configuration is stable, formation operations. The program plans to develop with all of the expected drawings releasable to additional software builds to correct these issues. manufacturing. However, the program has reported a delay in awarding the contract for one of these software Production Maturity builds, which could affect its ability to begin The C-130 AMP program reported that it operational testing in fiscal year 2012 as planned. demonstrated critical manufacturing processes using production-representative articles prior to In February 2010, the C-130 AMP program reported a entering production, but it did not assess Nunn-McCurdy unit cost breach of the significant manufacturing readiness levels at key suppliers and cost growth threshold after the average installation facilities. Program officials reported that procurement unit cost increased by almost 18 they will perform these assessments during low-rate percent over its existing baseline. According to the production and develop manufacturing maturity program office, the cost breach was due to the plans as needed in order to demonstrate its omission of training devices and adequate spares in readiness to begin full-rate production. We did not initial program estimates, inflation, delays in the assess the overall production maturity of the program’s production decision, and a change in the program because it does not collect statistical kit installation strategy. The program has been process control data on its critical manufacturing restructured, and the completion of operational processes. However, the program does track quality testing and low-rate production kit installation, as metrics related to the numbers of nonconformance well as the full-rate production decision, are now and corrective action reports, as well as the scheduled to occur more than 1 year later than percentage of inspection points passed. previously planned.

Other Program Issues Program Office Comments After 2 years of delays and the threat of program The program office concurred with this assessment. cancellation, the C-130 AMP received approval to enter production in June 2010. The first two aircraft scheduled for AMP kit installation began the modification process in August and October 2010, respectively. During low-rate production, the program will conduct a full and open competition to select a second contractor. The program will qualify the second contractor by providing installation and maintenance training, and the second contractor will perform up to five low-rate production kit installations. This contractor will compete with the current contractor for the full-rate production contract, which includes the procurement of 198 kits and up to 141 kit installations. The competition and

Page 52 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: CH-53K CH-53K - Heavy Lift Replacement The Marine Corps’ CH-53K helicopter will perform heavy-lift assault transport of armored vehicles, equipment, and personnel to support operations deep inland from a sea-based center of operations. The CH-53K program is expected to replace the legacy CH-53E helicopter and improve range and payload, survivability and force protection, reliability and maintainability, and coordination with other assets, while reducing total ownership cost.

Source: © 2008 Sikorsky Aircraft Corporation.

Concept System development Production

Program Development Design GAO Low-rate Initial Full-rate start start review review decision capability decision (11/03) (12/05) (7/10) (11/10) (9/14) (9/18) (3/19)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Sikorsky Aircraft As of Latest Percent Corporation 12/2005 07/2010 change Program office: Patuxent River NAS, Research and development cost $4,313.6 $5,915.4 37.1 MD Procurement cost $11,997.9 $15,986.9 33.2 Funding needed to complete: Total program cost $16,311.5 $21,902.3 34.3 R&D: $3,703.2 million Program unit cost $104.561 $109.512 4.7 Procurement: $15,986.9 million Total quantities 156 200 28.2 Total funding: $19,690.1 million Acquisition cycle time (months) 119 153 28.6 Procurement quantity: 196

According to program officials, both CH-53K Attainment of Product Knowledge current critical technologies are nearing maturity Production,288 and are expected to be fully mature by its design, and technology production decision in 2014. A third technology, maturity the viscoelastic lag damper, has been replaced e with a modified version of an existing technology to reduce cost and weight. The program Design and technolo192gy completed its design review in July 2010—16 maturity months later than planned—with over 90 percent Projection of its total expected design drawings released. Desired level of knowledg Developmental testing and initial operational capability have been delayed, and the overall cost Technology of development has increased by $1.7 billion. The maturity 96 program is revising its acquisition strategy to move the start of production up by 1 year to align with its production decision. The program will also update its schedule and cost estimate. As it stands now, delivery of the capability will occur 0 Development DOD GAO Production more than 2 years later than planned. start design review decision (12/05) review (11/10) (9/14) (7/10) Page 53 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: CH-53K CH-53K Program Delays in the CH-53K program may result in the extended use of and increased costs for legacy systems, such as the CH-53E and CH-53D helicopter. Technology Maturity Currently deployed CH-53E aircraft are flying at The CH-53K program began development in 2005 almost three times the planned utilization rate. This with three critical technologies, all of which were operational pace is expected to result in higher immature. One of these technologies, the airframe and component repair costs, as the Marine viscoelastic lag damper, was replaced by a modified Corps seeks to minimize CH-53E inventory version of an existing technology to reduce cost and reductions until CH-53K deliveries reach meaningful weight. According to the program office, the  levels. According to program officials, all available CH-53K’s two remaining critical technologies—the decommissioned CH-53E and CH-53D helicopters main rotor blade and the main gearbox—were have been reclaimed and all available parts have nearing maturity when the program’s design review been salvaged to keep the current inventory of was held in July 2010. The program office plans for aircraft in service. However, as a cost-saving these technologies to be fully mature and measure, the Marine Corps now plans to begin demonstrated in a realistic environment prior to its retiring the entire CH-53D fleet earlier than production decision in 2014. anticipated.

Design Maturity In 2008, the program office was directed to increase The CH-53K design appears stable. In July 2010, the the number of planned CH-53K aircraft from 156 to program office completed its critical design review 200 to accommodate an increase in Marine troop with over 90 percent of total expected design levels from 174,000 to 202,000. The quantity increase drawings released. However, the continuing added $5 billion in procurement cost to the program. maturation of critical technologies could result in design changes as testing progresses. Program Office Comments In its comments on a draft of this assessment, the Other Program Issues Navy stated that, during 2010, the CH-53K program CH-53K developmental testing and initial completed the critical design review and began operational capability have been delayed, and the assembly of the engineering development model test overall cost of development has increased by  articles. Critical technologies are maturing as $1.7 billion. To avoid further cost and schedule planned in the approved technology maturation problems, the program has taken several steps, plan. In August 2010, the Director, Defense Research including allowing more time to review the design and Engineering, certified that both critical and deferring certain requirements. For example, technology elements had achieved technology the program delayed its critical design review to readiness level 6, which is the appropriate level of ensure that all subsystems had completed their maturity for this stage of program development. The design reviews before moving forward. In addition, President’s 2011 budget fully funded the program to the program has received approval to defer three net achieve a fiscal year 2018 initial operational ready capabilities—variable message format (VMF), capability. The Navy also provided technical mode 5, and link 16—to later in production to comments, which were incorporated as appropriate. reduce development costs. According to the program office, deferring these capabilities would save approximately $103.5 million in the near term. However, it will also decrease the initial capability that is delivered to the warfighter. The program is currently revising its acquisition strategy to move the start of production up by 1 year to align with its production decision. Currently, there is a 1-year gap between the production decision date and the scheduled start of production. According to the program office, it is also updating its schedule and cost estimates.

Page 54 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: CVN 21 CVN 21 Future Aircraft Carrier The Navy’s CVN 21 program is developing a new class of nuclear-powered aircraft carriers. The carriers will include advanced propulsion, aircraft launch and recovery, and survivability technologies designed to improve operational efficiency, enable higher sortie rates, and reduce manpower. The Navy awarded a contract for detail design and construction of the lead ship, CVN 78, in September 2008 and expects delivery of the ship by September 2015. The Navy plans to award a construction contract for the second ship, CVN 79, in December 2012.

Source: U.S. Navy.

Concept System development Production

Program Construction- Production Lead-ship GAO Construction contract First ship Initial start preparation decision fabrication start review award—second ship delivery capability contract award (6/00) (5/04) (7/07) (9/08) (11/10) (12/12) (9/15) (9/16)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Northrop Grumman As of Latest Percent Shipbuilding-Newport News 04/2004 06/2010 change Program office: Washington, DC Research and development cost $4,732.3 $4,588.0 -3.1 Funding needed to complete: Procurement cost $30,316.2 $29,597.8 -2.4 R&D: $991.2 million Total program cost $35,048.5 $34,185.7 -2.5 Procurement: $19,047.6 million Program unit cost $11,682.827 $11,395.246 -2.5 Total funding: $20,038.8 million Total quantities 3 3 0.0 Procurement quantity: 2 Acquisition cycle time (months) 137 149 8.8

CVN 78 began construction in September 2008. Attainment of Product Knowledge However, 7 of the program’s 13 critical Production,288 technologies are still not fully mature because design, and technology they have not been demonstrated in a realistic, at- maturity sea environment. Of these technologies, the e electromagnetic aircraft launch system (EMALS), advanced arresting gear, and dual band radar Design and technolo192gy present the greatest cost and schedule risk. The maturity ship’s three-dimensional product model was completed in November 2009, but the contractor Desired level of knowledg is making design changes and could experience more as EMALS and other systems complete Technology testing. Seventy-two percent of the ship’s maturity 96 structural units are complete, accounting for about 19 percent of the total production hours. A number of units are behind schedule due to late materials. The program’s shift from a 4- to 5-year build cycle could increase costs if it results in the 0 Contract Lead ship GAO type of inefficiencies predicted by the shipbuilder. award fabrication review (5/04) (9/08) (11/10)

Page 55 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: CVN 21

CVN 21 Program Production Maturity The Navy awarded the CVN 78 construction contract in September 2008. Construction of approximately Technology Maturity 65 percent of the ship’s structural units is complete. Seven of the CVN 21 program’s 13 current critical These units account for about 19 percent of the technologies have not been demonstrated in a ship’s total production hours. As of July 2010, realistic, at-sea environment. Of these technologies, construction of the hull in dry dock was behind EMALS, the advanced arresting gear, and dual band schedule because of late material deliveries from radar present the greatest risk to the ship’s cost and suppliers. schedule. Program officials stated that EMALS development has been one of the primary drivers of Other Program Issues CVN 78 cost increases. Problems have occurred in In 2010, the CVN 21 program shifted from a 4- to 5- EMALS testing which could result in more design year build cycle, which could increase program changes later in the program. Testing uncovered a costs. According to program officials, the crack in the motor, which has already resulted in shipbuilder projects that this change will increase several design changes; and in January 2010, a motor costs by 9 to 15 percent due to the loss of learning controller software error caused damage to the and effect on the supplier base, among other EMALS hardware. Both fixes have successfully been inefficiencies. The Navy disagrees with this retested. The program completed the first four  assessment and reported to Congress that the shift F/A-18E launches in December 2010. The advanced will have minimal negative consequences. The dual arresting gear is nearing maturity and has completed band radar also presents cost risks for the program. extended reliability testing. However, delays in land- Program officials are considering buying the radar based testing with simulated and live aircraft could for both CVN 79 and CVN 80 at the same time, in lead to late delivery. The Navy finalized a fixed-price order to reduce the risks associated with the production contract for EMALS and the advanced production line being idle for up to 5 years. arresting gear in June 2010. Although the Navy However, this strategy could lead to increased costs continues to pay design and testing costs, any if changes identified during at-sea testing on CVN 78 EMALS changes identified during development will need to be incorporated into the already-procured be incorporated into the production units at no cost systems for the two follow-on ships. to the government. The dual band radar, which includes the volume search and multifunction Program Office Comments radars, is being developed by the DDG 1000 destroyer program and is also nearing maturity. In commenting on a draft of this assessment, the However, as a part of a program restructuring, the Navy generally concurred with this assessment. DDG 1000 eliminated the volume search radar from Officials stated the program is addressing the the program. According to Navy officials, radar technology and construction challenges for a development has not been affected, but CVN 78 will successful September 2015 delivery, and that CVN 79 now be the first ship to operate with this radar. is on track to award a construction contract by the Radar equipment will be delivered for installation first quarter fiscal year 2013. The Navy stated that and testing beginning September 2011 for the while the change from a 4- to 5-year build cycle will multifunction radar and in January 2012 for the increase the unit cost of the CVN 78 class carrier, it volume search radar. facilitates a reduced average yearly funding requirement over a longer period of time. The Navy Design Maturity also provided technical comments, which were incorporated as appropriate. In September 2008, CVN 78 began production with only 76 percent of its three-dimensional product model complete. The three-dimensional product model was completed by November 2009, but the contractor is currently making design changes to prevent electrical cable routing from interfering with other design features. As EMALS and other systems complete testing, additional design changes may be necessary.

Page 56 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: DDG 1000 DDG 1000 Destroyer The Navy’s DDG 1000 destroyer is a multimission surface ship designed to provide advanced land- attack capability in support of forces ashore and littoral operations. Construction has begun on the first and second ships, and the Navy anticipates awarding a construction contract for the third ship in the second quarter of fiscal year 2011. Bath Iron Works will build all three ships in this class with key segments built by Northrop Grumman Shipbuilding Gulf Coast.

Source: U.S. Navy.

Concept System development Production

Program Development Production Contract award— Lead ship Second ship GAO Initial start start decision construction construction construction review capability detail design start start (1/98) (3/04) (11/05) (8/06) (2/09) (3/10) (11/10) (6/16)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: BAE Systems, Bath As of Latest Percent Iron Works, Northrop Grumman 01/1998 01/2011 change Shipbuilding, Raytheon Research and development cost $2,240.4 TBD NA Program office: Washington, DC Procurement cost $32,043.5 TBD NA Funding needed to complete: Total program cost $34,283.9 TBD NA R&D: TBD Program unit cost $1,071.372 TBD NA Procurement: TBD Total quantities 32 TBD NA Total funding: TBD Acquisition cycle time (months) 128 221 72.7 Latest cost data resulting from the June 2010 Nunn-McCurdy restructuring were not available at the Procurement quantity: TBD time of this assessment.

The second ship in the DDG 1000 class began Attainment of Product Knowledge construction in March 2010 with a complete Production,288 design. While all 12 of the program’s critical design, and technology technologies are now nearing maturity or are fully maturity mature, 8 of these technologies will not be e demonstrated in a realistic environment until after installation on the first ship. Software Design and technolo192gy development for the total ship computing maturity environment also continues to be a challenge. In

fiscal year 2008, the Navy truncated the DDG 1000 Desired level of knowledg program to three ships, triggering a critical Nunn- McCurdy cost breach and a restructure of the Technology program. DOD removed the volume search radar maturity 96 from the baseline design and will modify software for the remaining multifunction radar to meet volume search requirements. The restructured program delayed initial operational capability by 1

year to allow additional time for the program to 0 Contract Lead ship GAO retire remaining software and production risks. award fabrication review (8/06) (2/09) (11/10)

Page 57 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: DDG 1000

DDG 1000 Program Other Program Issues In fiscal year 2008, the Navy truncated the DDG 1000 program to three ships, triggering a Nunn-McCurdy Technology Maturity unit cost breach of the critical threshold and a Three of DDG 1000’s 12 critical technologies are restructure of the program. To reduce program mature, and the remaining 9 have been costs, DOD removed the volume search radar from demonstrated in a relevant environment. The Navy the design, leaving only the multifunction radar on plans to fully demonstrate the integrated deckhouse the ship. According to program officials, removing before installation on the ship, but the remaining 8 the volume search radar will save the program $300 technologies will not be demonstrated in a realistic million and will not preclude DDG 1000 from environment until after ship installation. The design meeting its key performance parameters. However, review for one of the technologies—the ship’s long- the software for the multifunction radar will have to range land-attack projectile—was delayed from 2010 be modified to provide a volume search capability to 2011 to allow time to correct issues found during that meets all planned threat scenarios. The program rocket motor testing, but program officials noted office has not yet estimated the cost of these that the projectile has performed well and met multifunction radar modifications; it does not accuracy and range requirements in flight tests expect them to affect the program’s schedule. completed to date. The total ship computing According to program officials, the ship could environment (phased over six releases and one accept the volume search radar in the future spiral) is now nearing maturity, and, according to because space and weight will be reserved, but there program officials, the integration and testing of are currently no plans to include it. The program software release 5 is complete. However, software restructure also delayed initial operational development challenges remain. According to the capability by 1 year to the third quarter of fiscal year Defense Contract Management Agency (DCMA), 2016 to allow additional time for the program to there has been significant cost growth due to testing retire remaining software and production risks. The delays for release 5, and several unresolved program expects all three ships to be operational by problems have been deferred to release 6. DCMA 2018. has reported that these deferred requirements, coupled with software requirements changes for Program Office Comments release 6, could create significant cost and schedule In commenting on a draft of this assessment, the challenges. Navy stated that the program received milestone B approval, after the critical Nunn-McCurdy breach, in Design Maturity October 2010 and is closely monitoring and The DDG 1000 design appears stable. The design managing risk through comprehensive program was 88 percent complete at the start of lead ship metrics, program reviews, and an earned value construction and 100 percent complete shortly management system. At the time of the review, all thereafter. critical technologies had been at the appropriate level of maturity for the program phase. Earned Production Maturity value assessments of both shipbuilders and an The first DDG 1000 began construction in February independent logistics assessment are to be 2009 and the Navy estimates that approximately 30 completed in fiscal year 2011. All 26 major mission percent of the ship is complete. Fabrication of the systems equipment are in production and on track second ship began in March 2010, and 38 percent of for on-time delivery to the shipyard. Software the units that make up the ship are now in various release 6 is on track to support land-based testing stages of production. The Navy reported that it for the propulsion system and light off of the main contractually requires the shipbuilders to specify engine. The first advanced gun system magazine was detailed structural attributes to be monitored during delivered on time and the first gun has been shipped unit fabrication and integration in order to reduce for testing. A successful test mission readiness the risk of rework. While the shipbuilders are not review and associated tests for the multifunction currently meeting some of the production metrics, radar were completed in September 2010. The Navy program officials reported that these issues have also provided technical comments, which were been addressed in part by retraining personnel. incorporated as appropriate.

Page 58 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: E-2D AHE E-2D Advanced Hawkeye (E-2D AHE) The Navy’s E-2D AHE is an all-weather, twin-engine, carrier-based aircraft designed to extend early warning surveillance capabilities. It is the next in a series of upgrades the Navy has made to the E-2C Hawkeye platform since its first flight in 1971. The key objectives of the E-2D AHE are to improve target detection and situational awareness, especially in the littorals; support theater air and missile defense operations; and provide improved operational availability for the radar system.

Source: U.S. Navy.

Concept System development Production

Program/ Design Low-rate GAO Full-rate Initial Last development start review decision review decision capability procurement (6/03) (10/05) (5/09) (11/10) (12/12) (10/14) (2019)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Northrop Grumman As of Latest Percent Corp. 06/2003 06/2010 change Program office: Patuxent River, MD Research and development cost $3,784.3 $4,230.8 11.8 Funding needed to complete: Procurement cost $10,750.2 $13,556.9 26.1 R&D: $276.5 million Total program cost $14,534.5 $17,830.7 22.7 Procurement: $12,237.2 million Program unit cost $193.794 $237.743 22.7 Total funding: $12,528.7 million Total quantities 75 75 0.0 Procurement quantity: 65 Acquisition cycle time (months) 95 136 43.2

The E-2D AHE was approved for entry into Attainment of Product Knowledge production in May 2009 with all its critical Production,288 technologies mature and its design stable. We did design, and technology not assess production maturity; however, maturity according to E-2D program and Defense Contract e Management Agency officials, the contractor is performing well on a variety of production metrics Design and technolo192gy and inspections have not identified any significant maturity concerns. The program must complete a second

operational assessment and improve radar Desired level of knowledg reliability before it can award its next production contract. According to program officials, the Navy Technology completed a second operational assessment in maturity 96 November 2010. However, the program’s test plan for improving the reliability of the radar system remains aggressive. The program also experienced delays in development testing and the

delivery of pilot production aircraft related to a 0 Development DOD Production GAO now-resolved problem with the engine mount start design decision review design. (6/03) review (5/09) (11/10) (10/05) Page 59 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: E-2D AHE E-2D AHE Program March 2011 Defense Acquisition Board review of the program’s progress. According to program officials, the Navy completed a second operational Technology Maturity assessment in November 2010. However, the According to the Navy, all five of the E-2D AHE’s program’s test plan for improving the reliability of critical technologies are mature. The Navy the radar system remains aggressive. The radar must completed a technology readiness assessment in demonstrate a reliability rate greater than or equal to 2009 to support the program’s low-rate initial 65 hours. As of November 2010, the program production decision, and DOD concurred with that reported a radar reliability rate of 46.7 hours. assessment. The assessment included one new Program officials expect that the radar will exceed critical technology—the high-power UHF circulator. the 65-hour threshold by the March review because In the assessment, DOD raised concerns about the some corrective actions have already been UHF transmitter’s durability and its potential effect implemented, several fixes for identified root causes on life-cycle costs and operational availability. are waiting to be implemented, and few new types of According to program officials, the durability of the errors are occurring. According to program officials, parts has improved as a result of increased quality forthcoming software updates should address a control efforts. number of existing failures and improve reliability.

Design Maturity The program also experienced delays in The E-2D AHE design is stable. Program officials development testing and the delivery of pilot said that all current design drawings are releasable, production aircraft related to a now-resolved but some design changes will be necessary to problem with the engine mount design discovered incorporate recent modifications to the aircraft, during carrier suitability testing. Specifically, engine including those related to an engine heat shield movement led to a temperature sensor puncturing a puncture issue discovered during carrier suitability heat shield and making contact with a bulkhead testing. during simulated aircraft carrier landings. In response, certain carrier landing tests were stopped Production Maturity and other flight tests reduced from October 2009 We did not assess production maturity; however, through July 2010 while the program implemented a according to E-2D program and Defense Contract new engine mount design to address the problem. Management Agency officials, the contractor is The program office decided to adopt this design performing well on a variety of postproduction modification and others on the program’s three pilot metrics, and inspections have not identified any production aircraft, which resulted in a 3- to 4- significant concerns. The contractor reports month delay in the delivery of each aircraft. monthly to the Defense Contract Management According to program officials, the third pilot Agency and the program office on a series of production aircraft was delivered in November 2010. production metrics, such as scrap and rework rates, and the program office reported that the contractor Program Office Comments is meeting its rework goal. The program did not In commenting on a draft of this assessment, the identify any critical manufacturing processes program office provided technical comments, which associated with the E-2D AHE, nor does the program were incorporated as appropriate. require the contractor’s major assembly site to use statistical process controls to ensure its critical processes are producing high-quality and reliable products because components are assembled using manual processes that do not lend themselves to such measures.

Other Program Issues The E-2D AHE program must complete a second operational assessment and improve radar reliability before it can award its next production contract. The program plans to award this contract after a

Page 60 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Excalibur Excalibur Precision Guided Extended Range Artillery Projectile The Army’s Excalibur is a family of global positioning system–based, fire-and-forget, 155 mm cannon artillery precision munitions intended to provide improved range and accuracy. The near- vertical angle of fall is expected to reduce collateral damage, making it more effective in urban environments. The Army plans to develop the unitary warhead version in three increments—Ia-1, Ia-2, and Ib. We assessed increments Ia-1 and Ia-2 and made observations on increment Ib.

Source: U.S. Army.

Concept System development Production

Program/ Design Low-rate GAO Initial Last development review decision— review capability— procurement start increment Ia-2 increment Ia-2 (5/97) (5/05) (7/07) (11/10) (6/11) (2016)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Raytheon As of Latest Percent Program office: Picatinny Arsenal, NJ 02/2003 11/2010 change Funding needed to complete: Research and development cost $754.2 $975.1 29.3 R&D: $67.2 million Procurement cost $3,951.7 $695.8 -82.4 Procurement: $285.6 million Total program cost $4,705.9 $1,670.9 -64.5 Total funding: $352.8 million Program unit cost $.061 $.237 286.2 Procurement quantity: 3,930 Total quantities 76,677 7,050 -90.8 Acquisition cycle time (months) 136 171 25.7

Excalibur increments Ia-1 and Ia-2 are in Attainment of Product Knowledge production. According to program officials, their Production,288 critical technologies are mature and designs are design, and technology stable. The program received approval to begin maturity production of increment Ia-1 in May 2005 to e support an urgent requirement in Iraq and Afghanistan. Increment Ia-2 entered production in Design and technolo192gy July 2007 and completed initial operational test in maturity February 2010. After a design and prototype demonstration phase, the Army began engineering Desired level of knowledg and manufacturing development for increment Ib in August 2010. The two critical technologies for Technology this increment are mature. In May 2010, the Army maturity 96 reduced overall program quantities from 30,000 to 6,264 based on a review of precision munition needs. The resulting unit cost increase led to a Nunn-McCurdy breach of the critical threshold.

The program expects to be certified to continue in 0 Development DOD Production GAO early 2011. start design decision review (5/97) review (7/07) (11/10) (5/05) Page 61 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Excalibur

Excalibur Program Other Program Issues The Excalibur program is following an incremental acquisition strategy. Increment Ia-1 Excalibur was Technology Maturity fielded in Iraq and first used in combat in 2007. The The Excalibur’s three critical technologies for program office reported that over 85 percent of the increments Ia-1 and Ia-2—the airframe, guidance rounds expended in combat operations have system, and warhead—are mature. According to the functioned as expected. The program plans to program office, the technologies were demonstrated complete production of increment Ia-1 in fiscal years in a realistic environment at the time of their 2010 and 2011. Increment Ia-2 is currently in respective design reviews in May 2005 and March production and completed initial operational test 2007. After an 18-month prototype design and and evaluation in February 2010. During operational demonstration phase, the Army began engineering tests, it demonstrated an overall reliability rate of 73 and manufacturing development for increment Ib in percent by successfully completing 35 of 48 shots. August 2010. According to program officials, the two The increment Ib projectile, which is planned to critical technologies for this increment—the increase reliability and lower unit costs, is guidance systems and safe-and-arm fuze—are scheduled to begin production in fiscal year 2012. mature. The contractor for increment Ib plans to leverage existing technology from the increment Ia In May 2010, the Army reduced overall program program. quantities from 30,000 to 6,264 based on a review of precision munition needs. The resulting unit cost Design Maturity increase—from $47,000 to $99,000 per projectile— The Excalibur increment Ia-1 and Ia-2 designs are led to a Nunn-McCurdy breach of the critical stable. According to the program office, more than threshold. Congress was notified of the breach in 90 percent of each increment’s expected design August 2010. The program expects to be drawings were releasable at the time of their design restructured and certified by the Secretary of reviews. The number of design drawings increased Defense to continue in January 2011. As a result of by almost 20 percent between increment Ia-1 and  the Nunn-McCurdy breach, the program projects the Ia-2. According to a program official, the increase full-rate production decision for increment Ia-2 will was due to parts changes on increment Ia-1, as well move from August 2010 to February 2011. The as upgrades and changes for increment Ia-2. effects of the program restructure on increment Ib are still being determined by the program office. Production Maturity The Excalibur program appears to have overcome a Program Office Comments series of quality lapses that increased program costs, The program office provided technical comments on halted deliveries, and delayed the qualification of the a draft of this assessment, which were incorporated Ia-2. As a result of those problems, the program as appropriate. manager asked the contractor to review acceptance procedures and implement processes to control product quality. The program also qualified a new supplier for the inertial measurement unit—a part of the projectile’s guidance system—which has improved program reliability. While we could not assess Excalibur’s overall production maturity because statistical process controls have not been implemented at the system level, the program is taking steps to utilize these controls at the assembly plants and subcontractors. The contractor has started to compile these data and, as production continues and quantities increase, plans to look for key areas at the subcontractor level to place under control.

Page 62 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: EFV Expeditionary Fighting Vehicle (EFV) The Marine Corps’ EFV is designed to transport troops from ships offshore to inland locales at higher speeds and from longer distances than its predecessor, the Assault Amphibious Vehicle 7A1 (AAV 7A1). The EFV will have two variants—a troop carrier for 17 combat-equipped Marines and 3 crew members and a command vehicle to manage combat operations. Since the program started, DOD has also awarded contracts to redesign key subsystems to improve reliability and to develop an armor kit to protect EFVs from improvised explosive devices.

Source: U.S. Marine Corps.

Concept System development Production

Program Nunn-McCurdy Development GAO KP-2: Initial Operational Low-rate Initial start breach start review demonstrated testing decision capability reliability (12/00) (2/07) (7/08) (11/10) (1/11) (7/11) (1/12) (8/16)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: General Dynamics As of Latest Percent Program office: Woodbridge, VA 12/2000 11/2010 change Funding needed to complete: Research and development cost $1,625.2 $3,740.2 130.1 R&D: $575.7 million Procurement cost $7,299.8 $10,208.4 39.8 Procurement: $9,993.4 million Total program cost $9,018.7 $14,043.7 55.7 Total funding: $10,637.8 million Program unit cost $8.799 $23.682 169.2 Procurement quantity: 573 Total quantities 1,025 593 -42.1 Acquisition cycle time (months) 138 257 86.2

The EFV’s critical technologies are mature, but its Attainment of Product Knowledge design is still evolving. In 2007, DOD extended Production,288 system development, and the program revised its design, and technology approach to meeting its reliability requirements. maturity In addition to reliability, the program is monitoring e risks related to its schedule, the vehicle’s weight, and the potential for increased unit costs. Seven Design and technolo192gy new prototypes, which incorporate significant maturity design changes, are now undergoing development

and reliability growth testing, and the program Desired level of knowledg plans to demonstrate the prototypes’ initial reliability in January 2011. The Secretary of Technology Defense has proposed canceling the program. If it maturity 96 is not cancelled, the program will determine whether schedule or quantity changes—such as delaying its production decision or reducing initial quantities—are warranted.

0 Development DOD GAO Production start design review decision (12/00) review (11/10) (1/12) (12/08) Page 63 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: EFV EFV Program this program, stating that the EFV would be an enormously capable vehicle if completed, but that the mounting costs of acquiring it needed to be Technology Maturity weighed against other priorities. According to the program office, all four EFV critical technologies—high-pressure jet, high-speed planing, The program is currently monitoring four risk areas lightweight armor, and power diesel—are mature. that could affect the ultimate success of the program. Reliability growth has been identified as a Design Maturity risk because there is a chance that the design The EFV’s design is still evolving. In 2007, DOD changes the program has made may not be extended system development, and the program significant enough to provide the needed revised its approach to meeting its reliability improvement in reliability. The program has also requirements. One part of this approach involved a identified vehicle weight as a risk. Program officials restructured development effort to test redesigned expect aggressive weight management throughout components on existing prototypes. Another part the current development effort and low-rate initial involved building seven new prototypes, which production to mitigate this risk. While the program’s incorporate 180 significant subsystem design current weight assessment shows that the prototype changes to improve the EFV’s ability to move, shoot, design will not accommodate the required weight communicate, and carry and protect troops. The growth for future upgrades and increased loads, the initial reliability goal of the design changes is to program currently projects that the low-rate initial increase system reliability from the 4.5 hours mean production design will meet the weight growth time between operational mission failures measured margin in the EFV’s requirement document for in the program’s 2006 operational assessment to 16.4 production. The program’s schedule leading up to hours prior to the next operational assessment, the program’s production decision also faces risks. which is planned for July 2011. The prototypes are Specifically, technical and software issues could now undergoing development and reliability growth delay key events, such as developmental testing and testing, and the program plans to demonstrate this the start of the program’s July 2011 operational 16.4 hour goal in January 2011. The eventual goal is assessment. Finally, there is a risk that redesign of for low-rate initial production vehicles to the EFV could increase unit costs as well as demonstrate 43.5 hours of reliability during initial operations and support costs for the program. If the operational test and evaluation, which is scheduled program continues, it will determine whether to begin in July 2015. These operational test results schedule or quantity changes—such as delaying its will support the program’s full-rate production production decision or reducing initial quantities— decision in September 2016. are warranted to address these risks and its overall affordability. Production Maturity According to the EFV program, it is too early to Program Office Comments determine the maturity of the EFV production DOD provided technical comments on a draft of this processes. While the seven developmental prototype assessment, which we incorporated as appropriate. vehicles were built using tooling and processes that were representative of those used in production, the program does not intend to collect data on key manufacturing processes or use statistical process controls until low-rate production begins. EFV suppliers are performing inspections of the program’s key product characteristics and recording the data in preparation for future statistical process control analysis.

Other Program Issues The EFV program is entering a period that could determine whether or not it continues. In January 2011, the Secretary of Defense proposed canceling

Page 64 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JSF F-35 Lightning II (Joint Strike Fighter) DOD’s JSF program is developing a family of stealthy, strike fighter aircraft for the Navy, Air Force, Marine Corps, and U.S. allies, with the goal of maximizing commonality to minimize life-cycle costs. The carrier-suitable variant will complement the Navy F/A-18E/F. The Air Force variant will primarily be an air-to-ground replacement for the F- 16 and the A-10, and will complement the F-22. The short take-off and vertical landing variant will replace the Marine Corps F/A-18 and AV-8B aircraft.

Source: 2010 Lockheed Martin.

Concept System development Production

Program Development Design Production GAO Initial Initial Last start start review decision review capability— capability— procurement USMC USAF & USN (11/96) (10/01) (2/06 & 6/07) (6/07) (11/10) (12/12) (4/16) (2035)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Lockheed Martin As of Latest Percent Program office: Arlington, VA 10/2001 08/2010 change Funding needed to complete: Research and development cost $38,402.1 $53,663.1 39.7 R&D: $8,359.6 million Procurement cost $170,372.1 $229,467.6 34.7 Procurement: $215,146.0 million Total program cost $210,557.6 $283,674.5 34.7 Total funding: $223,815.5 million Program unit cost $73.467 $115.456 57.2 Procurement quantity: 2,385 Total quantities 2,866 2,457 -14.3 Acquisition cycle time (months) 116 174 50.0 Latest cost data do not fully account for cost and schedule changes resulting from the program’s critical Nunn-McCurdy unit cost breach.

The JSF is in production but three critical Attainment of Product Knowledge technologies are not mature, manufacturing Production,288 processes are not proven, and testing is not design, and technology complete. Continuing manufacturing maturity inefficiencies, parts problems, and technical e changes indicate that the aircraft’s design and production processes may lack the maturity Design and technolo192gy needed to efficiently produce aircraft at planned maturity rates. With most of developmental and operational flight testing still ahead, the risk of future design Desired level of knowledg changes is significant. DOD restructured the JSF program in February 2010 to address development Technology challenges. The projected cost growth triggered a maturity 96 Nunn-McCurdy unit cost breach of the critical threshold. According to program officials, the JSF is tracking well against its new, less aggressive test schedule despite late deliveries of test aircraft and lower than expected availability rates for short 0 Development DOD Production GAO take-off/vertical landing test aircraft. start design start review (10/01) review (6/07) (11/10) (2/06 and 6/07) Page 65 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JSF JSF Program incorporated additional software resources. The projected cost growth—including almost $104 billion since 2007—triggered a Nunn-McCurdy unit Technology Maturity cost breach of the critical threshold. A milestone The JSF program entered system development in review was scheduled for November 2010 to update 2001 with none of its eight critical technologies fully cost and schedule estimates. mature. According to the program office, five of these technologies are now mature and three According to program officials, the JSF is making technologies—mission systems integration, progress when measured against its new, less prognostics and health management, and radar—are aggressive test schedule and all three variants have nearing maturity. However, significant development had their first flights. However, several issues could risks remain as the program integrates and tests affect testing. The program had only delivered eight these technologies. aircraft to test sites as of December 2010, and short take-off/vertical landing test aircraft have Design Maturity experienced lower than expected availability rates. The JSF program did not have a stable design at its The program also continues to experience critical design reviews. The program has now challenges in developing and integrating the very released over 99 percent of its total expected large and complex software requirements needed to drawings; however, the program continues to achieve JSF capabilities. Further delays in either experience numerous design changes. With most of flight testing or software development could developmental and operational flight testing still jeopardize the Marine Corps’ planned initial ahead, the risk of future design changes and their operating capability date. Finally, the uncertain potential effect on the program are significant. fidelity of test results is a risk because the program relies on an unaccredited network of test Production Maturity laboratories and simulation models to evaluate Despite beginning production in 2006 and procuring system performance. 58 aircraft to date, the JSF program’s manufacturing processes are still not mature and only 12 percent of Program Office Comments its critical processes are in statistical control. DOD In commenting on a draft of this assessment, the has reduced near-term production quantities. program office noted that JSF is undergoing a However, continuing manufacturing inefficiencies, technical baseline review of requirements to parts problems, and technical changes indicate that complete the development effort as part of the the aircraft’s design and production processes may consideration for recertification of the development lack the maturity needed to efficiently produce milestone. Eight aircraft of the 10 anticipated in 2010 aircraft at planned rates. Managing an extensive, have been delivered to the test sites. An additional 4 still-maturing global network of suppliers adds are projected to be delivered by June 2011. The test another layer of complexity to producing aircraft program has slightly exceeded the overall test flight efficiently and on-time. The prime contractor is and test point metrics planned for 2010; testing of implementing manufacturing process the Marine Corps variant is behind plan while testing improvements. However, due to the extensive of the Air Force variant has exceeded plans. Mission amount of testing still to be completed, the program systems testing is underway with Block 1.0 on both could be required to make alterations to its Air Force and Marine Corps mission systems test production processes, changes to its supplier base, aircraft. Over half of the projected airborne system and costly retrofits to produced and fielded aircraft, software is in testing including the foundational if problems are discovered. sensor fusion architecture. Survivability testing has begun (live fire testing and radar cross section Other Program Issues signature ground testing) and results thus far are After an extensive programwide review, DOD matching predictions. The first airborne dynamic restructured the JSF program in February 2010 to signature test with aircraft AF-3 will begin address development challenges. The restructure December 2010. increased time and funding for system development, added more aircraft to support flight testing, reduced near-term procurement quantities, and

Page 66 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: FAB-T Family of Advanced Beyond Line-of-Sight Terminals (FAB-T) The Air Force’s FAB-T will provide a family of satellite communications terminals for airborne and ground-based users. FAB-T will address current and future communications capabilities and technologies, replacing many program-unique terminals. FAB-T is being developed incrementally. The first increment will provide voice and data military satellite communications for nuclear and conventional forces as well as airborne and ground command posts, including the B-2, B-52, RC-135, E-6, and E-4 aircraft. We assessed this increment.

Source: Boeing.

Concept System development Production

Development Critical GAO Low-rate Full-rate Last start design review review decision production procurement (9/02) (1/09) (11/10) (TBD) (TBD) (TBD)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Boeing As of Latest Percent Program office: Hanscom AFB, MA 12/2006 12/2009 change Funding needed to complete: Research and development cost $1,514.4 $1,735.2 14.6 R&D: $220.2 million Procurement cost $1,627.0 $2,194.6 34.9 Procurement: $2,081.8 million Total program cost $3,141.5 $3,929.9 25.1 Total funding: $2,302.0 million Program unit cost $14.544 $16.172 11.2 Procurement quantity: 209 Total quantities 216 243 12.5 Acquisition cycle time (months) 129 NA NA The program did not provide an updated cost position or future funding stream because of ongoing changes related to the rebaseline.

The FAB-T program expected to enter production Attainment of Product Knowledge in February 2010 with its critical technologies Production,288 mature and its design stable; however, the design, and technology program now plans to significantly extend its maturity development phase to more fully develop the high- e data-rate variant and reduce the concurrency in testing and production. A new low-rate production Design and technolo192gy decision date has not yet been approved, but is maturity tentatively scheduled for the first quarter of fiscal

year 2013. Two critical technologies have not yet Desired level of knowledg demonstrated their maturity as planned, and the FAB-T program office continues to monitor Technology certification of the system’s cryptography by the maturity 96 National Security Agency. The FAB-T design also does not appear to be stable; however, we were unable to specifically assess it because the program has not provided updated information on Not assessed its design relating to its restructure and rebaseline 0 Development DOD GAO Production efforts. start design review decision (9/02) review (11/10) (TBD) (1/09) Page 67 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: FAB-T FAB-T Program office, hardware qualification problems and testing failures made this level of concurrency an unacceptable risk. The new low-rate production Technology Maturity decision date is tentatively scheduled for November The FAB-T program expected to enter production in 2012. In addition, in response to cost and schedule February 2010 with all six critical technologies growth on the FAB-T program, the Office of the mature and demonstrated in a realistic environment. Secretary of Defense directed the establishment of However, according to program officials, two critical four integrated product teams to perform reviews technologies—the continuous transverse stub similar to those required for a Nunn-McCurdy antenna and the high-data-rate software breach on management, technical, and cost issues, configuration architecture—have not yet and to examine potential alternative sources and demonstrated their maturity as planned. The changes to requirements. program has decided to extend the development phase, in large part to more fully develop the high- FAB-T certification by the National Security Agency data-rate software variant. FAB-T’s critical (NSA) is another key step in the program. FAB-T technologies were not assessed at development start needs to properly protect information at various in 2002 because it was not yet a major defense classification levels and NSA will provide a acquisition program. certification of the cryptography in certain equipment. In June 2009, NSA completed a review of Design Maturity the low-data-rate version of system software and The FAB-T design does not appear to be stable. Even approved limited use of the FAB-T cryptographic though the program office reported a high element in program testing. Program officials percentage of releasable design drawings last year, expected to be authorized to test the extended-data- there have been changes to the program since then rate version of system software around the end of that could affect the design. Specifically, as a result 2010. The NSA is currently scheduled to complete of hardware qualification problems and testing final certification of this version in March 2013. failures, the program decided to extend However, delays in the maturation of the high-data- development and delay production. Resolving these rate software configuration architecture technology issues could require design changes. According to could affect the certification schedule. program officials, the program also anticipates that two engineering changes—one related to secure Program Office Comments transmissions and another related to environmental The Air Force provided technical comments, which specifications—will require additional design work. were incorporated as appropriate. We were unable to specifically assess the design as a whole because the program has not provided updated information on its design relating to its restructure and rebaseline efforts.

Other Program Issues The FAB-T program has recently been restructured and rebaselined to more fully develop the high-data- rate variant and reduce concurrency between testing and production. The program delayed its scheduled February 2010 production decision and plans to extend its development phase. In January 2009, the contractor delivered the first FAB-T engineering model. According to program officials, FAB-T completed all the objectives for developmental flight testing of the hardware for the low-data-rate system in August 2009. At that time, program officials expected the extended or high-data-rate system to undergo most of its testing concurrently with low- rate production. However, according to the program

Page 68 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Global Hawk Global Hawk (RQ-4A/B) The Air Force’s Global Hawk is a high-altitude, long- endurance unmanned aircraft with integrated sensors and ground stations providing intelligence, surveillance, and reconnaissance capabilities. The Global Hawk will replace the U-2. After a successful technology demonstration, the system entered development and limited production in March 2001. The program includes RQ-4A aircraft similar to the original demonstrators, as well as larger and more capable RQ-4Bs. We assessed the RQ-4B, which is being procured in three blocks.

Source: Northrop Grumman.

Concept System development Production

Demonstration Development start / GAO Last program start low-rate decision review procurement (2/94) (3/01) (11/10) (2018)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Northrop Grumman As of Latest Percent Program office: Wright-Patterson AFB, 03/2001 07/2010 change OH Research and development cost $1,026.3 $3,948.7 284.8 Funding needed to complete: Procurement cost $4,255.0 $9,481.6 122.8 R&D: $1,067.5 million Total program cost $5,312.4 $13,575.7 155.5 Procurement: $5,339.1 million Program unit cost $84.323 $176.307 109.1 Total funding: $6,406.6 million Total quantities 63 77 22.2 Procurement quantity: 39 Acquisition cycle time (months) 55 TBD NA

The Global Hawk RQ-4B has mature critical Attainment of Product Knowledge technologies, a stable design, and proven Production,288 production processes, but it remains at risk for design, and technology late design changes and costly retrofits. The maturity completion of operational tests for the aircraft e that make up the largest part of the program has been delayed nearly 4 years by testing discoveries, Design and technolo192gy concurrent testing, resource constraints, and maturity weather problems. The program will have procured more than half of those aircraft by the Desired level of knowledg time testing is complete in December 2010. The program also plans to procure more than half the Technology aircraft with advanced radar before it completes maturity 96 operational testing in 2013. The Air Force is taking steps to address some of the testing delays. In fiscal year 2010, the Air Force increased the total number of aircraft to be procured from 54 to 77 and extended planned production through 2018. 0 Development DOD Development GAO start design start/production review (NA) review decision (11/10) (NA) (3/01) Page 69 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Global Hawk Global Hawk Program found in operational testing, it could result in costly retrofits for large numbers of aircraft. The Global Hawk program has continued to experience delays Technology Maturity in developmental and operational testing. The The critical technologies for the RQ-4B are mature. completion of operational tests for the block 20 and However, the program must still successfully test 30 aircraft has been delayed nearly 4 years to two key capabilities—the advanced signals December 2010. The start of operational testing for intelligence payload and multiple platform radar—to block 40 aircraft has been delayed by more than 3 ensure they perform as expected. The first flight of years to March 2013. According to the Global Hawk’s an RQ-4B equipped with the signals intelligence December 2009 Selected Acquisition Report, several payload occurred in September 2008 and operational factors contributed to the most recent schedule testing is scheduled to be completed in December slips, including developmental test discoveries; 2010. After delays in its development, the first flight concurrent development and production testing; of an RQ-4B equipped with the multiple platform testing resource constraints; and weather problems. radar is expected to occur in April 2011. According to program officials, a shift in focus and Development testing is underway. resources required to address a Joint Urgent Operational Need, using two block 20 aircraft, has Design Maturity also contributed to block 40 operational test delays. The RQ-4B basic airframe design is stable with all of The Air Force is taking steps to address some of the its expected design drawings released; however, the testing delays. For example, the program is now program remains at risk for late design changes and conducting aircraft acceptance tests at Beale Air costly retrofits if problems are discovered in testing. Force Base in order to free up resources for During the first year of production, frequent operational testing at Edwards Air Force Base. substantive engineering changes increased development and airframe costs and delayed Program Office Comments deliveries and testing. Substantial commonality In commenting on a draft of this assessment, the Air between the RQ-4A and RQ-4B had been expected, Force emphasized that the Global Hawk program but as the designs were finalized and production has improved program execution while reducing geared up, the design differences were much more program risk. The Air Force noted that older RQ-4A extensive and complex than anticipated. Global Hawk aircraft—which we did not assess— have been successfully used by the warfighters and Production Maturity other government agencies to carry out various The manufacturing processes for the RQ-4B missions. The service also noted that each of the airframe are mature and in statistical control. In variants of its larger RQ-4B aircraft is now either in addition, the program reports that it is meeting its operations or testing. Flight operations of deployed quality goal on the number of nonconforming parts. aircraft and flight testing of the advanced radar The RQ-4B aircraft is being produced in three payload are expected to begin in 2011. The Air Force configurations. Block 20 aircraft are equipped with noted that current challenges facing the program an enhanced imagery intelligence payload; block 30 include initial system deployments and aircraft have both imagery and signals intelligence normalization of operations and sustainment. In payloads; and block 40 aircraft will have an addition to commenting on this assessment, the Air advanced radar surveillance capability. All six block Force provided technical comments, which we 20 aircraft have been produced. Production incorporated where appropriate. continues on block 30 and block 40 aircraft. The first block 30 aircraft was delivered in November 2007 and the first block 40 aircraft was delivered without the sensor in November 2009.

Other Program Issues The Global Hawk program expects to have procured all of its block 20 aircarft, and more than half of its block 30 and block 40 aircraft before operational testing is complete. As a result, if problems are

Page 70 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: GPS-IIIA Global Positioning System (GPS) IIIA The Air Force’s Global Positioning System (GPS) III program will develop and field a new generation of satellites to supplement and eventually replace GPS satellites currently in use. It consists of three increments: IIIA, IIIB, and IIIC. Other programs will develop the ground control system and user equipment. We assessed GPS IIIA, which intends to provide capabilities such as a stronger military navigation signal to improve jamming resistance and a new civilian signal that will be interoperable with foreign satellite navigation systems.

Source: Lockheed Martin.

Concept System development Production

Development Design GAO Production First satellite start review review decision available for launch (5/08) (8/10) (11/10) (12/10) (5/14)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Lockheed Martin As of Latest Percent Program office: El Segundo, CA 05/2008 07/2010 change Funding needed to complete: Research and development cost $2,486.9 $2,615.2 5.2 R&D: $1,238.6 million Procurement cost $1,396.3 $1,409.6 1.0 Procurement: $1,409.6 million Total program cost $3,883.1 $4,024.8 3.6 Total funding: $2,648.2 million Program unit cost $485.392 $503.099 3.6 Total quantity: 8 Total quantities 8 8 0.0 Acquisition cycle time (months) NA NA NA We could not calculate acquisition cycle times for GPS IIIA because initial operational capability will not occur until GPS IIIC satellites are fielded.

The GPS IIIA program completed its critical Attainment of Product Knowledge design review in August 2010 with its critical Production,288 technologies mature and design stable. The design, and technology program plans to prove its production processes maturity by building and testing a prototype spacecraft e prior to its December 2010 production decision. This prototype will include almost all satellite Design and technolo192gy parts excluding redundant units, but will not be maturity flight-worthy. A complete GPS IIIA satellite will

not be available for testing prior to the production Desired level of knowledg decision. The GPS IIIA program is using a “back- to-basics” approach, which emphasizes best Technology practices such as maintaining stable requirements maturity 96 and using mature technologies. The program still faces risks to delivering and launching satellites as planned, due to its compressed schedule and dependence on a separately developed ground control system being fully functional. 0 Development DOD GAO Production start design review decision (5/08) review (11/10) (12/10) (8/10) Page 71 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: GPS-IIIA GPS-IIIA Program While this approach should enable the GPS III program to deliver satellites more quickly than the predecessor GPS program, its schedule is still Technology and Design Maturity aggressive considering the complexities associated The GPS IIIA program’s critical technologies are with the integration phase. The Air Force plans to mature and its design is stable. The critical launch the first GPS IIIA satellite in 2014. This would technologies have changed for the GPS IIIA program require the program to go from contract award to as it has developed its design. Prior to contract first launch 3.5 years faster than the GPS IIF. award, the program’s five critical technologies were based on a notional government architecture for the In addition, the GPS IIIA program could be affected satellite. According to the program office, the by the development schedule for the next-generation Lockheed Martin satellite design differs significantly GPS ground control system, the OCX, which is being from this architecture. A postpreliminary design managed as a separate major defense acquisition review technology readiness assessment in 2009 program. Though the GPS IIIA satellites can provide identified seven critical technologies. However, the positioning and timing services without OCX, it is number of critical technologies was revised to eight needed to control other features of the satellites, when the results of that assessment were finalized in such as the enhanced military signal and additional 2010. All eight have been demonstrated in a relevant civil signals. Until OCX is operational, these environment. According to the program office, the additional signals cannot be operated on the GPS number of critical technologies has been stable as of IIIA satellite, and the Air Force is reluctant to launch the postpreliminary design review assessment. In the second IIIA satellite before the first one is fully addition, the design for GPS IIIA is stable with 98 tested. The Air Force currently plans to deliver GPS percent of design drawings releasable at its August OCX Block I in August 2015—15 months after the 2010 critical design review. first planned GPS IIIA satellite launch.

Production Maturity Program Office Comments The GPS IIIA program plans to reduce risk and In commenting on a draft of this assessment, GPS prove out its production processes by building and program officials acknowledged that there is testing a prototype spacecraft prior to its December currently a disconnect between the OCX delivery 2010 production decision. However, this prototype schedule and the GPS IIIA launch schedule. As a will not be production-representative. It will include result, the program office recently awarded a almost all satellite parts, excluding redundant units. contract to the OCX and GPS IIIA contractors to According to the program office, it will not be flight- study possible technical solutions to provide worthy because its parts will not go through the preliminary ground control capabilities to support flight screening process. A complete GPS IIIA the first GPS IIIA launch. The program expects this satellite will not be available for testing prior to the interim system to be delivered in the third quarter of production decision. We did not assess production 2013. The program officials believe that this system maturity because the program office does not collect will provide the capability to launch and check out statistical process control data for its critical the GPS IIIA vehicle ahead of OCX completion. manufacturing processes, but rather uses other Program officials also provided technical comments, process and technology maturity metrics. which were incorporated as appropriate.

Other Program Issues The GPS IIIA program has adopted an acquisition approach that should increase its chances of meeting its cost and schedule goals; however, the program still faces risks that could affect the on-time delivery and launch of GPS satellites. GPS IIIA is being managed using a “back-to-basics” approach, which is designed to maintain stable requirements, implement an incremental development strategy, use mature technologies, and provide more oversight than under the previous GPS satellite program.

Page 72 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: GPS OCX GPS III OCX Ground Control Segment The Air Force’s next generation GPS control segment (OCX) will provide command, control, and mission support for the GPS Block II and III satellites. OCX is expected to assure reliable and secure delivery of position and timing signals to serve the evolving needs of GPS military and civilian users. The Air Force plans to develop OCX in four blocks to deliver upgrades as they become available. We assessed the first block, which will support the operations of GPS Block II and Block III satellites.

Source: U.S. Air Force.

Concept System development Production

Program GAO Preliminary Development Production start review design review start decision (2/07) (11/10) (4/11) (6/11) (4/15)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Raytheon Latest Percent Program office: El Segundo, CA As of 07/2010 change Funding needed to complete: Research and development cost NA $2,693.1 NA R&D: $1,675.5 million Procurement cost NA $22.6 NA Procurement: $22.5 million Total program cost NA $2,891.3 NA Total funding: $1,873.6 million Program unit cost NA $1,445.642 NA Procurement quantity: 0 Total quantities NA 2 NA Acquisition cycle time (months) NA NA NA

The GPS OCX program is scheduled to enter Attainment of Product Knowledge development in June 2011 with its 14 critical Production,288 technologies nearing maturity. In February 2010, design, and technology the Air Force awarded a cost-reimbursement maturity contract to Raytheon for Blocks I and II of the e OCX program. The GPS OCX program built prototypes and plans to hold a preliminary design Design and technolo192gy review in April 2011 prior to entry into engineering maturity and manufacturing development, as required by

DOD acquisition policy and statute. The GPS OCX Desired level of knowledg will not be fielded in time for the May 2014 launch of the first GPS IIIA satellite. As a result, the GPS Technology Directorate is considering funding a parallel effort maturity 96 that accelerates existing launch and checkout requirements to develop a command and control capability for the first GPS IIIA satellites. Not Not assessed assessed 0 GAO Development DOD Production review start design decision (11/10) (6/11) review (4/15) (TBD) Page 73 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: GPS OCX GPS OCX Program program understood the risks associated with the development effort before moving forward to the next phase of the program. Technology Maturity According to program officials, when the GPS OCX The Air Force plans to deliver GPS OCX Block I in enters development in June 2011, its 14 critical August 2015—15 months after the first planned GPS technologies will be nearing maturity. As part of its IIIA satellite launch. To address this issue, the GPS risk-reduction activities, the program selected two Directorate is considering funding a parallel effort contractors, through a competitive process, to that accelerates existing launch and checkout develop system-level prototypes. It also plans to requirements to develop a command and control hold a preliminary design review in April 2011 prior capability for the first GPS IIIA satellites. However, to entry into engineering and manufacturing GPS Directorate officials indicated that the effort development, as required by DOD acquisition policy would not enable new capabilities offered by GPS and statute. In an October 2010 memorandum, the IIIA satellites, including a military signal designed to Director, Defense Research and Engineering, stated enable resistance to jamming and three civil signals. that an independent assessment of the program The program expects to release a request for found that all but one critical technology had been proposal for the parallel effort during the first demonstrated in a relevant environment. The quarter of fiscal year 2011 and receive the Air technology that had not been demonstrated in a Force’s authority to proceed during the third quarter relevant environment has been deferred to future of fiscal year 2011. OCX blocks. The assessment also found that GPS OCX will eventually require larger bandwidths, and Program Office Comments questioned whether the backup command and The GPS Directorate provided written technical control site will have the capability to take control of comments that were incorporated as appropriate. all the functions managed by the primary site. Furthermore, the assessment found that the program did not have a security architecture that meets all information assurance requirements, and that the OCX system may not be able to handle large data sets required to service external users. The Director recommended that the Air Force conduct a technology readiness assessment on future OCX blocks to explain how these and any new requirements are fully addressed by mature technologies.

Other Program Issues In February 2010, the Air Force awarded a cost- reimbursement contract to Raytheon for Block I and II of the GPS OCX program. According to program officials, a cost-reimbursement contract was used because of the high level of risk associated with developing complex software programs for GPS OCX.

The GPS OCX program plans to enter engineering and manufacturing development in June 2011—over 2 years later than initially planned. According to program officials, this delay was due, in part, to the need to hold a preliminary design review and report on its results before the milestone review. In addition, the GPS Directorate and GPS OCX program manager wanted to make sure that the

Page 74 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Gray Eagle Gray Eagle The Army’s Gray Eagle, formerly known as ER/MP, will perform reconnaissance, surveillance, target acquisition, and attack missions. It will operate either alone or with other platforms such as the Longbow Apache helicopter. Each system includes 12 aircraft as well as ground control stations, ground and air data terminals, automatic takeoff and landing systems, and ground support equipment. The program consists of Block 1 systems and two less-capable Quick Reaction Capability systems. We assessed the Block 1 configuration.

Source: General Atomics Aeronautical Systems, Inc.

Concept System development Production

Program/development Design Low-rate GAO Initial Full start review decision review capability capability (4/05) (11/06) (2/10) (11/10) (7/12) (8/12)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: General Atomics As of Latest Percent Program office: Huntsville, AL 04/2005 09/2010 change Funding needed to complete: Research and development cost $339.8 $875.3 157.6 R&D: $246.0 million Procurement cost $660.5 $2,965.8 349.0 Procurement: $2,285.1 million Total program cost $1,000.2 $4,844.0 384.3 Total funding: $3,374.3 million Program unit cost $200.046 $372.613 86.3 Procurement quantity: 10 Total quantities 5 13 160.0 Acquisition cycle time (months) 50 87 74.0 As of December 2010, DOD had not yet approved a new cost and schedule baseline for the program.

The Gray Eagle entered production in February Attainment of Product Knowledge 2010 without having all of its critical technologies Production,288 mature. The program office reported that the design, and technology system’s design is stable and its production maturity processes are proven, but the program remains at e risk for late and costly design and manufacturing changes during production until its critical Design and technolo192gy technologies have been fully integrated and maturity tested. A January 2010 risk review board identified risks related to areas including software, engine Desired level of knowledg availability, and supplier capacity. The Army has identified operational availability and reliability as Technology a risk after limited user testing showed that the maturity 96 system could not meet its key performance parameter for that area. The Army has plans in place to mitigate these risks and will undertake various risk-reduction activities leading up to the system’s entry into initial operational test and 0 Development DOD Production GAO evaluation in 2011. start design decision review (4/05) review (2/10) (11/10) (11/06) Page 75 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Gray Eagle

Gray Eagle Program Production Maturity According to an independent Army assessment of the program’s production readiness, for critical or Technology Maturity major suppliers, its manufacturing process maturity The Gray Eagle entered production in February 2010 was satisfactory and manufacturing infrastructure without all its critical technologies mature, as met or exceeded requirements for low-rate initial recommended in DOD’s Technology Readiness production. However, in January 2010, an Army Assessment Deskbook. Two technologies—the review board noted production risks associated with heavy fuel engine and deicing—have been assessed the tactical common data link subcontractor’s as mature. The other three technologies—the capacity to produce, provide spares for, and repair automatic takeoff and landing system, tactical that component as needed to meet program common data link, and manned-unmanned schedule. In addition, it identified issues with teaming—are nearing maturity. software development and engine availability resulting from issues with financial stability of the Prior to 2010, the program office had reported that engine supplier. According to the program office, the all its critical technologies would be mature at data link production capacity issue has been largely production. However, an independent technology mitigated and the prime contractor has qualified a readiness assessment by the Office of the Director, second engine supplier. Defense Research and Engineering, reached a different conclusion on both the identification and Other Program Issues maturity level of the program’s critical technologies. In October 2010, the Army identified operational This assessment resulted in the Army dropping one availability and reliability as a risk after limited user critical technology, adding two newly identified testing showed that the system could not meet its technologies, and downgrading the maturity level of key performance parameter for that area. The Army three technologies. According to the program office, plans to undertake risk-reduction activities leading the maturity levels were downgraded because the up to the system’s initial operational test and program office had previously assessed the evaluation in 2011. technologies alone, whereas the independent assessment considered their maturity when Program Office Comments integrated with Gray Eagle. For example, the program office had assessed the automatic takeoff In commenting on a draft of the assessment, the and landing system as mature because the same program office indicated that the DOD Technology technology is used on the already-fielded Shadow Readiness Assessment Deskbook discourages the unmanned aircraft system, but the independent practice of evaluating technology readiness based assessors rated it as nearing maturity because the on degree of integration. The program also believes system had not yet been fully integrated into the that our product knowledge graph did not accurately Gray Eagle and tested in an operational capture the Gray Eagle’s production maturity environment. because there are more methods to assess maturity than the critical processes assessment we used. The Design Maturity program did not detail the methods it believed applicable. Finally, the program stated that all risk- While the program office indicated that the Gray mitigation plans were on schedule as of January Eagle design is stable, the program remains at risk 2011. The program also provided technical for late and costly design and manufacturing corrections, which were incorporated as changes during production until its critical appropriate. technologies have been fully integrated and tested. Despite this risk, the Army plans to proceed with GAO Response production. In 2009, the Army’s Aviation and Missile Research, Development, and Engineering Center According to the DOD deskbook, technologies independently assessed the program’s production should be at technology readiness level (TRL) 7 or readiness and concluded that the design of the higher at production start. To achieve TRL 7, a system was mature and stable enough such that program should demonstrate a system prototype in potential design changes would not present a an operational environment, which would require significant risk to the program during low-rate initial them to be integrated in the system. production. Page 76 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Increment 1 E-IBCT Increment 1 Early-Infantry Brigade Combat Team (E-IBCT) The Army’s E-IBCT program will augment brigade- level capabilities through an incremental, expedited fielding of systems to current forces. The first increment, scheduled for fielding in late 2011, includes unattended sensors, unmanned ground and air vehicles, and new radios and battle command software. Increment 1 evolved from Army efforts to quickly equip current forces with the more mature capabilities from the now terminated Future Combat System program. The Army anticipates at least one follow-on increment.

Source: U.S. Army.

Concept System development Production

Program / Limited Design Low-rate Limited GAO Initial Full-rate Last development user review decision user review capability production procurement start test test (7/04) (9/09) (10/09) (12/09) (9/10) (11/10) (9/11) (12/11) (08/13)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Boeing As of Latest Percent Program office: Warren, MI 02/2010 08/2010 change Funding needed to complete: Research and development cost $590.0 $595.9 1.0 R&D: $112.9 million Procurement cost $2,594.2 $2,661.7 2.6 Procurement: $2,182.6 million Total program cost $3,184.2 $3,257.6 2.3 Total funding: $2,295.5 million Program unit cost $353.801 $361.956 2.3 Procurement quantity: 8 Total quantities 9 9 0.0 Acquisition cycle time (months) 27 27 0.0

E-IBCT Increment 1 was approved for production Attainment of Product Knowledge in December 2009, even though an independent Production,288 review team later found that none of its critical design, and technology technologies were mature, and the program was maturity still making design changes to address reliability e issues identified in testing. Since that time, the Army has worked to improve performance and Design and technolo192gy reliability of the E-IBCT systems. According to the maturity Army, all current Increment 1 critical technologies

are mature and its systems’ designs are stable. The Desired level of knowledg results of an updated independent technology assessment were not available at the time of our Technology review. In addition, the Army was unable to maturity 96 provide production data from the contractor. The Under Secretary of Defense for Acquisition, Technology and Logistics was scheduled to review the program in December 2010 to determine

whether to proceed with the production of the 0 Development DOD Production GAO next two sets of systems. start design decision review (NA) review (12/09) (11/10) (10/09) Page 77 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Increment 1 E-IBCT

Increment 1 E-IBCT Program Other Program Issues The Under Secretary of Defense for Acquisition, Technology and Logistics was scheduled to review Technology Maturity the program in December 2010 to consider the According to the Army, all current E-IBCT Increment systems’ readiness for further testing and fielding, 1 critical technologies are mature, although whether to proceed with additional production, and independent reviewers disagree. Prior to the the direction for the remainder of the program. The December 2009 production decision, the Army review was to be based on the Army’s progress reported that 9 out of 10 critical technologies were improving the systems’ reliability and network mature. However, a March 2010 independent performance during the September 2010 limited user assessment reported that none of the critical test. technologies were mature and only two technologies were nearing maturity. The assessment found that During tests leading up to the September 2010 the Army had not demonstrated two key radio limited user test, the Army reported that only one technologies under the expected operational system failed to meet its reliability requirement for conditions or at the required range. It also found that entrance into the limited user test. However, an other technologies displayed erratic performance, Army assessment of the 2010 limited user test experienced excessive reboots, or were relatively reported that only three of the five systems met or primitive with regard to efficiency and robustness. exceeded reliability requirements. Although According to Army officials, a September 2010 reliability did improve since the 2009 limited user limited user test planned to demonstrate the test, the systems were collectively assessed as not improved maturity of these technologies. To support providing force effectiveness at the system of the Under Secretary of Defense for Acquisition, systems level and, with the exception of the small Technology and Logistics’ December 2010 review of unmanned ground vehicle, provided minimal the program, the Director, Defense Research and military utility. Engineering, planned to complete an updated independent technology assessment. The results of Program Office Comments that assessment were not available at the time of our According to the program office, in 2009, significant review. concerns were raised regarding the reliability of Increment 1 systems and network maturity. Testing Design Maturity conducted in 2010 demonstrated significant According to the Army, the designs of the E-ICBT reliability improvements with all systems (less the Increment 1 systems are stable with 93 percent of unmanned aerial system) greatly exceeding their the total expected design drawings releasable to reliability requirements. Testing also proved manufacturing. These designs were not stable when network maturity for the E-IBCT configuration and the program received approval to enter production the Network Integration Kit was determined to be a in December 2009. The program has made 86 design key command and control enabler. The Training and changes since then to address performance and Doctrine Command identified two key issues with reliability issues. These design changes were the Network Integration Kit, and the program incorporated into the equipment that was used in a manager has already implemented and September 2010 limited user test and into the demonstrated fixes. The Training and Doctrine systems’ production configuration. Command has voiced strong support for the Network Integration Kit and Small Unmanned Production Maturity Ground Vehicle. On the basis of a December 2010 We did not assess production maturity because the Army Configuration Steering Board and a pending Army was unable to provide statistical process Defense Acquisition Board, a descoping of systems control data on critical manufacturing processes. and quantities is expected. Cost and quantity However, Army documents indicate that the systems information for the anticipated changes are have achieved an engineering manufacturing predecisional and were not made available for this readiness level of 3, which demonstrates readiness report. The Army provided additional technical for low-rate production. comments, which were incorporated as appropriate.

Page 78 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Scorpion Intelligent Munitions System-Scorpion The Army’s Intelligent Munitions System-Scorpion is a remotely controlled, antivehicular landmine alternative system. The Scorpion includes an integrated system of lethal and nonlethal munitions, sensors, software, and communications that detects, tracks, classifies, reports, engages, and kills light wheeled through heavy tracked vehicles. As part of the Army’s capability portfolio review, it was determined that the Scorpion is no longer affordable. Program closeout was approved by a configuration steering board in October 2010.

Source: U.S. Army.

Concept System development Production

Development Critical GAO Low-rate Initial operational start design review review decision capability (5/06) (4/09) (11/10) (12/11) (9/13)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Textron Defense Latest Percent Systems As of 11/2010 change Program office: Picatinny Arsenal, NJ Research and development cost NA $487.3 NA Funding needed to complete: Procurement cost NA $870.5 NA R&D: $78.7 million Total program cost NA $1,685.2 NA Procurement: $870.0 million Program unit cost NA $.642 NA Total funding: $1,275.9 million Total quantities NA 2,624 NA Procurement quantity: 2,624 Acquisition cycle time (months) NA 89 NA

The Scorpion program’s critical technologies are Attainment of Product Knowledge mature and its design is stable. The program Production,288 initially planned to use the Joint Tactical Radio design, and technology System, but has switched to the more mature maturity Spider radio. While the Scorpion’s design was not e stable at its April 2009 design review, over 90 percent of its design drawings are now releasable Design and technolo192gy and its critical software functionality has been maturity tested. The program will conduct a series of production readiness reviews as it prepares for its Desired level of knowledg December 2011 production decision. In addition, the program is already producing test hardware on Technology the production floor using production processes, maturity 96 personnel, and tooling. Originally part of the Future Combat System, the program was established as a stand-alone program in January 2007. The separation caused cost growth which led to the program being designated a major 0 Development DOD GAO Production defense acquisition program in February 2010. start design review decision (5/06) review (11/10) (12/11) (4/09) Page 79 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Scorpion Scorpion Program hardware with engineering oversight. All Scorpion hardware delivered for qualification testing has been produced on the production floor, using production Technology Maturity processes, personnel, tooling, and special test All four of the Scorpion’s critical technologies—the equipment. The program will also conduct a series control station computing unit, situational of production readiness reviews to support the awareness, antivehicle effects, and communications transition to production. through Spider radio—are mature. When the program began development in 2006, its critical Other Program Issues technologies included the Joint Tactical Radio The Scorpion began development as part of the System (JTRS) Cluster 5 radio and the JTRS Future Combat System (FCS) program in 2006. It Network Enterprise Domain soldier radio waveform. supports the National Landmine Policy announced As a result of differences between the Scorpion and in February 2004, which stated the United States JTRS development schedules, the program switched would no longer use non-self-destructing antivehicle to the mature Spider radio. and antipersonnel landmines after December 31, 2010. In January 2007, the Army separated the Design Maturity Scorpion program from the FCS. In February 2010, The Scorpion’s design has stabilized since its April due to development cost growth, which was 2009 design review when only 61 percent of its total attributed to negative effects from the separation expected drawings were releasable. Over 90 percent and technical issues during development, the of its design drawings are now releasable and its Scorpion program was designated as a major safety-critical and major software functionality has defense acquisition program. been tested. While risk-reduction tests completed in November 2009 have demonstrated the capability of As part of the Army’s capability portfolio reviews, it the system’s design utilizing production- was determined that the Scorpion is no longer representative hardware, the system’s poor affordable and that the Army is willing to accept the performance against heavy tracked vehicles and operational risk of not fielding this capability. another required target is a design concern. However, some of this technology will roll into the According to the program’s January 2010 post– Spider program. The decision to conduct an orderly critical design review assessment, the Army needs to closeout was approved by a configuration steering make a decision on the importance of this board in October 2010 and the official acquisition requirement because the program is utilizing decision memorandum is pending. resources to try to meet it and it could affect the system’s overall performance. In addition, the Program Office Comments assessment identified system reliability and an The Scorpion program was developed to avoid a aggressive program schedule as challenges. capability gap associated with the National Landmine Policy and field a capability prior to Production Maturity December 31, 2010. Due to this pressure, the We could not assess production maturity because program was initially schedule driven. Following the the program does not collect statistical process critical design review, the program plan was updated control data on its critical manufacturing processes; to reflect an event driven schedule. Following the however, the program has taken a number of steps risk-reduction testing in November 2009, design to prepare for its planned December 2011 changes and modifications to the requirement production decision. The program has identified its improved performance against the heavy tracked critical manufacturing processes and key product vehicles, and changes to the tactics, techniques, and characteristics and uses yield and defect data and procedures by the engineer school resulted in defect management to track them. According to the improved performance against lightwheeled program office, the contractor began identifying and vehicles. These enhancements were demonstrated developing custom tooling and test equipment that during conduct of development and live fire testing would be required for production shortly after in September 2010. Program officials also provided development start. Production processes and tooling technical comments, which were incorporated as were refined during this period and utilized in the appropriate. next phase wherein operations personnel built

Page 80 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JAGM Joint Air-to-Ground Missile (JAGM) The Joint Air-to-Ground Missile is a joint Army/Navy program with Marine Corps participation. The missile will be air-launched from helicopters and fixed-wing aircraft and designed to target tanks; light armored vehicles; missile launchers; command, control, and communications vehicles; bunkers; and buildings. It is to provide line-of-sight and beyond line-of sight capabilities and can be employed in a fire-and-forget mode or a precision attack mode. The missile will replace Hellfire, Maverick, and air- launched TOW.

Source: Department of Defense.

Concept System development Production

Technology Preliminary Prototype GAO Development Critical Production Initial operational development start design review missile tests review start design review start capability (9/08) (6/10) (9/10) (11/10) (TBD) (TBD) (TBD) (TBD)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: TBD Latest Percent Program office: Warren, MI As of 07/2010 change Funding needed to complete: Research and development cost NA $1,650.8 NA R&D: $1,206.9 million Procurement cost NA $5,202.1 NA Procurement: $5,202.1 million Total program cost NA $6,852.9 NA Total funding: $6,409.0 million Program unit cost NA $.202 NA Procurement quantity: 33,853 Total quantities NA 33,853 NA Acquisition cycle time (months) NA 69 NA

According to the program office, the three JAGM Attainment of Product Knowledge critical technologies are expected to be nearing Production,288 maturity and demonstrated in a relevant design, and technology environment before a decision is made to enter maturity development. However, an independent e technology readiness assessment identified five critical technologies, at least one of which has not Design and technolo192gy reached this level of maturity. The program office maturity has incorporated a provision in the draft request for proposal for the development contract that Desired level of knowledg may mitigate some of the technology risk by requiring both contractors to submit two rocket Technology motor designs. According to program officials, the maturity 96 release of the request for proposal has been delayed until the third quarter of fiscal year 2011 because the program’s acquisition strategy and requirements needed to be updated to reflect the Not Not assessed assessed cancellation of the Armed Reconnaissance 0 GAO Development DOD Production Helicopter and new guidance on affordability. review start design decision (11/10) (TBD) review (TBD) (TBD) Page 81 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JAGM JAGM Program engineering and manufacturing development contract between the two technology development contractors. Technology Maturity According to the program office, the three JAGM The Army and the Navy will continue to rely on critical technologies are expected to be nearing Hellfire and Maverick missiles until JAGM is fielded. maturity and demonstrated in a relevant The Army will continue to extend the fielding of environment before a decision is made to start Hellfire to meet the needs of the warfighter, while system development. The critical technologies the Navy will rely on both Maverick and Hellfire include a multimode seeker for increased until JAGM becomes available. countermeasure resistance, boost-sustain propulsion for increased standoff range, and a Program Office Comments multipurpose warhead for increased lethality. In commenting on a draft of this assessment, the However, an independent technology readiness program office stated that due to the delay in the assessment identified five critical technologies, at signing of the JAGM acquisition strategy, key dates least one of which has not reached this level of in the development phase have the potential to be maturity. The program office has incorporated a delayed. provision in the draft request for proposal for the engineering and manufacturing development contract that may mitigate some of the technology risk by requiring both contractors to submit two rocket motor designs.

Other Program Issues In September 2008, the Army awarded two fixed- price incentive contracts to Raytheon and Lockheed Martin for a 27-month JAGM technology development phase. During technology development, each contractor completed three tests using prototype missiles in order for the program to assess the technical risks of proceeding to the next phase of development. In addition to testing prototypes, each contractor completed a preliminary design review. According to program officials, a post–preliminary design review assessment should be complete by December 2010.

The JAGM program planned to receive approval to enter system development in November 2010 and award an engineering and manufacturing development contract in December 2010. However, the release of the request for proposal for this contract has been delayed because the program’s acquisition strategy and requirements needed to be updated to reflect the cancellation of the Armed Reconnaissance Helicopter, the addition of the  OH-58 Kiowa as a replacement platform, and new guidance on affordability. According to program officials, contract award is now expected no earlier than the third quarter of fiscal year 2011. The JAGM program office has requested a justification and approval for a limited competition for the

Page 82 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JASSM Joint Air-to-Surface Standoff Missile (JASSM) The Air Force’s JASSM program is intended to field a next-generation air-to-ground cruise missile capable of stealthy flight and reliable performance at affordable costs. It is designed to destroy enemy targets from outside the range of air defenses. The Air Force is currently producing a baseline JASSM and is developing an extended range version— JASSM-ER—that will more than double the range of the baseline version. The two variants are 70 percent common in hardware and 95 percent common in software. We assessed the JASSM-ER variant.

Source: U.S. Air Force.

Concept System development Production

Program JASSM ER GAO ER ER ER ER last start development development review low-rate full-rate initial procurement start start decision decision capability (6/96) (11/98) (6/03) (11/10) (12/10) (6/13) (TBD) (2025)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Lockheed Martin As of Latest Percent Program office: Eglin AFB, FL 11/1998 06/2010 change Funding needed to complete: Research and development cost $1,004.7 $1,462.3 45.5 R&D: $84.4 million Procurement cost $1,252.9 $5,738.7 358.0 Procurement: $4,736.2 million Total program cost $2,281.6 $7,201.0 215.6 Total funding: $4,820.7 million Program unit cost $.924 $1.435 55.3 Procurement quantity: 3,747 Total quantities 2,469 5,018 103.2 Acquisition cycle time (months) 75 87 16.0 Latest costs include funding for both JASSM and JASSM-ER. According to program officials, JASSM- ER does not have an approved program baseline separating its costs from the baseline program, but is completing one for the upcoming production decision.

According to the program office, the JASSM-ER Attainment of Product Knowledge plans to enter production in December 2010 with Production,288 all of its critical technologies and manufacturing design, and technology processes mature. We did not assess the design maturity stability of the JASSM-ER because the program e office does not collect design drawing data. However, the JASSM-ER design has been Design and technolo192gy demonstrated to perform as intended. It has been maturity successful in 10 out of 11 flight tests. The program plans to perform additional tests in order to Desired level of knowledg demonstrate that it meets its reliability requirement. As part of the upcoming production Technology decision, the program has assessed its maturity 96 manufacturing readiness and proven out manufacturing and quality processes in a pilot-line environment. The cost of the JASSM-ER could be higher than predicted because prior cost estimates were overly optimistic and flight tests will be 0 Development DOD GAO Production needed to achieve its reliability requirement. start design review decision (6/03) review (11/10) (12/10) (10/05) Page 83 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JASSM JASSM Program (combined JASSM and JASSM-ER production rate), which has not been achieved since 2005. Not producing at the expected rate has led to a less Technology Maturity efficient production process and a longer production An independent review panel recently assessed all period, both of which increase costs. Further, five JASSM-ER critical technologies as mature. The Lockheed Martin officials stated that low production five technologies are the engine system, engine lube rates could cause skilled labor to look elsewhere for system, fuse, low-observable features, and global work and JASSM reliability could be adversely positioning system. This assessment was conducted affected. In addition, according to the Air Force, as to support the upcoming JASSM-ER production many as 20 flight tests may be needed to fully decision. demonstrate JASSM-ER’s reliability goal of 85 percent. These flight tests could cost as much as  Design Maturity $70 million. According to the program office, the JASSM-ER design is stable. The design has been demonstrated Program Office Comments to perform as intended and the missile has been In commenting on a draft of this assessment, the successful in 10 out of 11 flight tests. However, we JASSM program office noted the reliability concerns did not specifically assess design stability because have been alleviated by successful tests of JASSM- the JASSM-ER program office does not track the ER (10 of 11, completed November 2010) and Lot 7 number of design drawings. According to the JASSM baseline (15 of 16, completed October 2009). program office, under the total system performance Additional flight tests beyond the budgeted responsibility arrangement that was in place when operational test and reliability assessment programs the program was initiated, all design drawings were are not required to achieve the reliability developed and managed by the contractor. The Air requirement of 85 percent following JASSM-ER Lot Force has since sought more control over the design 4. In fact, JASSM-ER is currently at 87 percent at the of the missile. It now has approval authority over conclusion of developmental tests. The risk for major configuration changes, as well as approval higher JASSM-ER costs stems from unforeseen authority over configuration changes that that may budget cuts that reduce production quantities and increase cost, require retrofit, or affect safety for drive up unit price. Additionally, the JASSM-ER missiles currently in production. design is stable as evidenced by the last five flight tests flown with the current production Production Maturity configuration, and the program’s successful According to a program official, the JASSM-ER plans completion of a production readiness review. The to enter production in December 2010 with all of its JASSM program also provided technical comments, manufacturing processes mature. The Air Force which were incorporated where appropriate. recently assessed JASSM-ER at a manufacturing readiness level 8, meaning, among other things, that its technologies are mature, manufacturing and quality processes and procedures have been proven in a pilot-line environment, and it is ready to enter into low-rate production. In addition, the JASSM-ER missiles are being produced on the same production line as the JASSM baseline, and the two missiles are 70 percent common in hardware and 95 percent common in software.

Other Program Issues The cost of the JASSM-ER program could be higher than predicted. First, lower than projected annual procurement levels could increase production costs. The Air Force’s current cost estimate for the JASSM program may be overly optimistic since it is based on a production rate of 280 missiles per year

Page 84 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JHSV Joint High Speed Vessel (JHSV) The JHSV is a joint Army and Navy program to acquire a high-speed, shallow-draft vessel for rapid intratheater transport of combat-ready units. The ship will be capable of operating without reliance on shore based infrastructure. The program intends to produce a total of 18 ships, 13 for the Navy, and 5 for the Army. DOD authorized construction of the lead ship in December 2009. It is expected to be delivered in November 2011.

Source: Austal USA.

Concept System development Production

Program Contract Lead-ship GAO Lead-ship Initial Second ship Last ship start award fabrication start review delivery capability delivery delivery (4/06) (11/08) (12/09) (11/10) (11/11) (12/12) (1/13) (1/17)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Austal, USA As of Latest Percent Program office: Washington, DC 02/2009 07/2010 change Funding needed to complete: Research and development cost $126.5 $125.7 -0.7 R&D: $18.8 million Procurement cost $3,456.0 $3,543.4 2.5 Procurement: $2,593.2 million Total program cost $3,582.5 $3,669.1 2.4 Total funding: $2,612.0 million Program unit cost $199.028 $203.836 2.4 Procurement quantity: 13 Total quantities 18 18 0.0 Acquisition cycle time (months) 48 50 4.2

The JHSV program entered production in Attainment of Product Knowledge December 2009 with its critical technologies Production,288 mature, but without a complete three dimensional design, and technology design. Nine of the ship’s 46 design zones were maturity complete in the three-dimensional model when e construction began. According to the Navy’s own measure of design maturity—which takes into Design and technolo192gy account other design metrics such as maturity completeness of two-dimensional design drawings and engineering reviews, as well as the three- Desired level of knowledg dimensional model—the design was at least 85 percent complete. Program officials state that the Technology three-dimensional model was completed in maturity 96 September 2010. As of October 2010, 33 modules were in production utilizing instructions derived from the model. Prior to starting production, DOD agreed to reduce the JHSV’s required transit speed, in order to avoid the need for a significant 0 Contract Lead ship GAO redesign that could have affected the program’s award fabrication review cost and schedule. (11/08) (12/09) (11/10)

Page 85 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JHSV

JHSV Program Other Program Issues DOD chose the JHSV to participate in the capital budget account pilot program, which was created to Technology and Design Maturity control cost growth by providing stable funding. The JHSV program awarded its detailed design and Under this initiative, the program office must gain construction contract in November 2008 with 17 of approval from the Joint Chiefs of Staff, the Office of its 18 critical technologies mature and demonstrated the Under Secretary of Defense for Acquisition, in a realistic environment. Before production began Technology and Logistics, and the Office of the in December 2009, the program was required to Under Secretary of Defense (Comptroller) for demonstrate that all technologies were mature. changes in funding or requirements. Program officials stated that this is useful as it allows them to In December 2009, DOD authorized the Navy to stabilize their requirements and the flow of work to begin construction of the lead ship without a the shipyard. Funding for 10 of the program’s 18 complete three-dimensional design. According to ships is currently guaranteed. program officials, 9 of the ship’s 46 three- dimensional design zones were complete at the start Prior to starting production, DOD agreed to reduce of construction. An additional 14 design zones were the JHSV’s required transit speed, to avoid design nearing completion. According to program officials, changes that could have affected the program’s cost construction start was delayed 34 days to complete and schedule. Previously JHSV had been required to product modeling for the JHSV’s most complex have a transit range of 4,700 nautical miles traveling areas, such as the ship’s machinery rooms. This level at a speed of 25 knots. According to program of design maturity falls short of GAO’s officials, transiting at this speed requires additional recommended shipbuilding best practices, which amounts of fuel that would have triggered the need call for achieving a complete and stable three- for a significant redesign, cost increases, and dimensional product model before construction schedule delays. As a part of a configuration steering begins. The program office believes that the board meeting, officials from the JHSV Navy completion of the model prior to construction start requirements office, Joint Staff, and DOD agreed was less critical for JHSV because it is not as that the speed could be reduced to 23 knots to complex as other Navy ships, such as the DDG 1000. preserve the current design and schedule with According to the Navy’s own measure of design minimal effect on meeting mission needs. maturity—which includes the completion of two- dimensional design drawings and engineering Program Office Comments reviews as well as the three-dimensional model—the The program office did not concur with our findings design was at least 85 percent complete. According related to design maturity. As certified by Navy and to program officials, the three-dimensional model Defense Department officials, per Public Law 110- was completed in September 2010. 181, greater than 85 percent of the design was completed prior to construction start by the Navy’s Production Maturity measure of design maturity. In addition, the three- Prior to the start of production, the JHSV program dimensional model was completed prior to the start was required to demonstrate that its manufacturing of fabrication of the future USS Vigilant, the first processes were in control. The program conducted JHSV, on September 13, 2010. Significant production two pilot production phases and built a pilot module and financial risk has been avoided by using proven in the shipbuilder’s new module manufacturing commercial production design and technology, facility. According to program officials, production ensuring stable requirements, minimizing change, will be monitored through the use of earned value and through the ruthless pursuit of cost reduction management data to track the cost of the work and efficiency. performed, and through reviews and inspections performed by the American Bureau of Shipbuilding GAO Response and the Navy’s Supervisor of Shipbuilding. As of Our findings on the design maturity of the JHSV are October 2010, the program office reported that 33 of based on metrics determined by previous audits of the ship’s modules were in production. Navy shipbuilding programs and commercial best practices.

Page 86 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JLENS Joint Land Attack Cruise Missile Defense Elevated Netted Sensor System (JLENS) The Army’s JLENS will provide over-the-horizon detection and tracking of land-attack cruise missiles and other targets. The Army is developing JLENS in two spirals. Spiral 1 is complete and served as a test bed to demonstrate the concept. Spiral 2 will utilize two aerostats with advanced sensors for surveillance and tracking, as well as mobile mooring stations, communication payloads, and processing stations. JLENS will provide surveillance and engagement support to other systems, such as  PAC-3, SM-6, and MEADS. We assessed Spiral 2.

Source: U.S. Army.

Concept System development Production

Development Design GAO Low-rate Initial Full-rate Last start review review decision capability decision procurement (8/05) (12/08) (11/10) (3/12) (9/13) (11/14) (2020)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Raytheon As of Latest Percent Program office: Redstone Arsenal, AL 08/2005 12/2009 change Funding needed to complete: Research and development cost $1,975.5 $2,154.7 9.1 R&D: $649.1 million Procurement cost $4,520.7 $5,041.9 11.5 Procurement: $5,041.9 million Total program cost $6,566.9 $7,378.0 12.4 Total funding: $5,852.8 million Program unit cost $410.432 $461.122 12.4 Procurement quantity: 14 Total quantities 16 16 0.0 Acquisition cycle time (months) 97 97 0.0

According to program officials, JLENS will enter Attainment of Product Knowledge production with mature technologies, a stable Production,288 design, and proven production processes. The design, and technology program began development in 2005 with only one maturity Projection of its five critical technologies mature, and only e two of the four current critical technologies are mature. The design appears stable, but the Design and technolo192gy potential for design changes remains until the maturity maturity of JLENS components have been

demonstrated. In September 2010, an aerostat Desired level of knowledg accident resulted in the loss of one of the JLENS platforms. This accident and other system Technology integration challenges are expected to delay maturity 96 several key program events, including the production decision. Twelve of the program’s 15 critical manufacturing processes are currently in control. The JLENS program has also completed a

number of key production planning activities, 0 Development DOD GAO Production such as assessing supplier capabilities and risks. start design review decision (8/05) review (11/10) (3/12) (12/08) Page 87 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JLENS

JLENS Program Other Program Issues The JLENS program is working to address several risks that could affect the program’s cost, schedule, Technology Maturity and performance. First, the program received $32 JLENS entered system development in August 2005 million less than the amount requested in the with only one of its five critical technologies mature. President’s fiscal year 2010 budget. If additional The program subsequently combined two of the funding is not provided in fiscal year 2012, the critical technologies—the communications payload program reports it will not be able to procure the and the processing group—into the communications equipment to field an initial operational capability by processing group. The communications processing the end of fiscal year 2013. Second, due to the group and platform are currently mature. The September 2010 aerostat accident and subsequent program expects to demonstrate the fire control loss of a platform, the program expects several key radar and surveillance radar in a realistic events, including the start of production, to be environment before the program enters production. delayed. Third, if problems occur during systems Many of the JLENS radar technologies have legacy integration and verification tests, the program components. However, sensor software items expects that cost and schedule would be affected. related to signal processing, timing, and control, as Fourth, if test site preparations are not complete by well as element measurement, are not yet mature. April 2011, then the production timeline could be The program office has successfully conducted tests jeopardized. Finally, the program could also be of the fire control radar antenna, but the integration affected by alignment with the Army’s Integrated Air of both the fire control radar and surveillance radar and Missile Defense program. As part of the components in the program’s system integration integrated strategy, the Army extended the system laboratory has yet to occur. development and demonstration phase by 12 months. The JLENS program is waiting approval of a Design Maturity new acquisition program baseline with updated cost The JLENS design appears stable, but the potential and schedule estimates that reflect this change. for design changes will remain until key JLENS components have been integrated and tested. For Program Office Comments example, a first flight demonstration of the aerostat In commenting on a draft of this assessment, the was successfully conducted in August 2009, but the program stated that the Army is planning to request program must still complete a series of tests funds in its fiscal year 2012 budget to offset the fiscal integrating the JLENS mobile mooring station with year 2010 reduction. The program also reported the aerostat. In September 2010, the program experiencing development challenges that have experienced the loss of a platform following an caused system integration delays, and schedule aerostat accident. The program is analyzing the challenges due to a September 2010 aerostat cause of the accident, as well as other system accident. The program office continues to work on a integration issues. The JLENS program has received new acquisition program baseline. A new cost approval to transport the mobile mooring station estimate was presented to the Army Cost Review without armor, which mitigates a risk the program Board in July 2009. The estimate will be updated office has identified in the past. based on the results of an Army review and submission of the President’s fiscal year 2012 Production Maturity budget. A revised baseline is expected to be The JLENS program projects that it will enter approved in the third quarter of fiscal year 2011. The production with all 15 of its critical manufacturing Army provided technical comments, which were processes mature and stable. According to the incorporated as appropriate. program office, 12 of the program’s critical manufacturing processes are currently in control. The JLENS program has also completed a number of key activities that are essential to effective production management, including updating its manufacturing plan and addressing areas such as supplier capabilities and risks, cost, quality control, materials, producibility, and workforce skills.

Page 88 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JPALS Joint Precision Approach and Landing System (JPALS) JPALS is a joint Army, Navy, and Air Force program that will replace the obsolete radar-based SPN-46 and SPN-35 systems. It is a Global Positioning System/Inertial Navigation System-based system that will provide a rapidly deployable, adverse weather, adverse terrain, day-night precision approach and landing capability for all DOD ground and airborne systems. Increment 1A is a Navy-led sea-based ship system, and increment 1B will integrate JPALS with sea-based aircraft. We assessed increment 1A.

Source: Department of Defense.

Concept System development Production

Development Preliminary GAO Critical Production Initial Full-rate start design review design decision capability decision review review (7/08) (12/09) (11/10) (12/10) (2/13) (12/14) (6/15)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Raytheon As of Latest Percent Program office: Lexington Park, MD 07/2008 07/2010 change Funding needed to complete: Research and development cost $780.2 $750.3 -3.8 R&D: $305.2 million Procurement cost $210.1 $219.0 4.2 Procurement: $219.2 million Total program cost $997.1 $976.2 -2.1 Total funding: $524.4 million Program unit cost $26.950 $26.384 -2.1 Procurement quantity: 26 Total quantities 37 37 0.0 Acquisition cycle time (months) 75 77 2.7

JPALS began system development in July 2008 Attainment of Product Knowledge with both of its critical technologies nearing Production,288 maturity. JPALS is primarily a software design, and technology development effort but also includes commercial maturity hardware components. The hardware design is e stable and program officials accepted the system’s drawings in preparation for the December 2010 Design and technolo192gy critical design review. However, design stability maturity has been affected by requirements changes. As of

January 2011, there were 387 requirements in the Desired level of knowledg system performance specification—an increase of 33 since the start of development. Officials also Technology report ship integration challenges on CVN 78 may maturity 96 require changing the antenna placement to Projection accommodate performance and maintenance requirements. The program plans to enter production in 2013. Increment 1B will begin Not assessed development in 2012 and integrate the system with 0 Development GAO DOD Production the avionics of the F/A-18E/F, EA-18G, and  start review design decision MH-60R/S. (7/08) (11/10) review (2/13) (12/10) Page 89 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JPALS

JPALS Program Other Program Issues According to program officials, several ship integration challenges are being addressed. Technology Maturity Specifically, the current JPALS antenna location for The JPALS program began development in July 2008 the JPALS system on CVN 78 affects the program’s with two critical technologies—the geometry extra ability to meet performance and maintenance redundant almost fixed solution and the vertical requirements. Trade studies are investigating several protection level / lateral protection level—nearing potential antenna location changes to determine the maturity. Program officials expect both critical optimal position for it. The cost effect of moving the technologies to be mature and demonstrated in a antenna will not be known until the studies are realistic environment by the JPALS production complete. Program officials also continue to monitor decision in 2013. While JPALS utilizes existing the system’s maintainability to ensure JPALS commercial components for most of its hardware, requires no manpower increase compared to legacy its functionality will be enabled by over 700,000 lines systems—a key performance parameter. JPALS has of software code. The program plans to rely heavily completed a detailed maintenance analysis, the on reused code with 77 percent of the program’s results of which indicate that the estimated total lines of code expected to be reused. If less workload meets this manpower requirement. JPALS software is reused than originally estimated, the is also at risk of exceeding its weight limit for  potential consequences are longer development time CVN 78. It currently exceeds the requirement by 500 and greater cost. pounds. The program office reported that CVN 78 has updated the ship design to account for the Design Maturity increased weight. JPALS is primarily a software development effort, but also includes commercial hardware The JPALS acquisition strategy separates the components. The hardware design is stable with 96 program into seven increments. Increment 1 is percent of the total expected design drawings separated into two phases, A and B. Increment 1B— released to manufacturing. The drawings cover the aircraft integration—will begin development in 2012 JPALS ship system-radio, antenna, receiver, racks, and integrate the system with the avionics of the and console. The program also tracks requirement F/A-18E/F, EA-18G, and MH-60R/S. Increment 2— changes to monitor design stability. As of January land-based—will be led and funded by the Air Force 2011, there were 387 requirements in the system and was expected to begin development during performance specification—an increase of 33 since 2011. the start of development. These changes are due to system design gaps uncovered for L-class ships, the Program Office Comments Joint Strike Fighter, and legacy landing systems and In commenting on a draft of this assessment, the updated maintenance requirements. According to Navy generally concurred with this assessment. program officials, detailed software requirements Officials stated that the JPALS increment 1A are stable and proceeding according to schedule to program is on track for system integration with support software development. The first of seven acceptable risk, that cost and schedule performance software blocks is complete and blocks 2 and 3 are are within the baseline plan, and that the system on schedule. requirements and acquisition strategy continue to be accurate, supportable, and executable. The Navy Production Maturity also provided technical comments, which were Program officials plan to employ various techniques incorporated as appropriate. to assess production maturity, including tool design, fabrication metrics, and quarterly production readiness reviews. The program will build eight engineering development models to be installed on aircraft carriers and sent to test facilities to demonstrate system performance. These models are expected to be delivered in fiscal years 2011 through 2012. The program plans to enter production in February 2013.

Page 90 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: AMF JTRS Airborne and Maritime/Fixed Station Joint Tactical Radio System (AMF JTRS) DOD’s JTRS program is developing software-defined radios that will interoperate with existing radios and AMF JTRS Small Airborne AMF JTRS Maritime – Fixed Station increase communications and networking capabilities. A joint program executive office provides a central acquisition authority that cuts across the military services. Program and product offices develop hardware and software for users with similar requirements. The AMF program will develop radios and associated equipment for integration into nearly 160 different types of aircraft, ships, and fixed stations.

Source: Department of Defense.

Technology/system development Initial capability

Program Design GAO SA production M/F production Initial start review review decision decision capability (3/08) (11/09) (11/10) (11/11) (6/12) (8/14)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Lockheed Martin As of Latest Percent Program office: San Diego, CA 10/2008 07/2010 change Funding needed to complete: Research and development cost $1,915.8 $1,998.1 4.3 R&D: $950.4 million Procurement cost $6,117.1 $6,213.7 1.6 Procurement: $6,213.7 million Total program cost $8,032.9 $8,211.8 2.2 Total funding: $7,164.1 million Program unit cost $.296 $.303 2.2 Procurement quantity: 26,878 Total quantities 27,102 27,102 0.0 Acquisition cycle time (months) 80 77 -3.8 The program office reported quantities in terms of channels rather than radios. The program is developing a 2-channel small airborne (SA) radio and a 4-channel maritime/fixed station (M/F) radio based on a single common architecture.

The AMF JTRS program completed its design Attainment of Product Knowledge review in November 2009 with its five critical Production,288 technologies nearing maturity and its design design, and technology stable. There will be an independent technology maturity readiness assessment before the small airborne e variant production decision, currently planned for November 2011. AMF JTRS production processes Design and technolo192gy are also approaching maturity with manufacturing maturity sites having demonstrated a capability to produce

components or subsystems in a production- Desired level of knowledg relevant environment. Each of the AMF variants will undergo initial operational test and evaluation Technology after the program’s initial production decision. maturity 96 AMF JTRS quantities could increase depending on whether the Navy and Marine Corps decide to acquire AMF JTRS small airborne radios for their networking capabilities. Not assessed 0 Development DOD GAO Production start design review decision (3/08) review (11/10) (11/11) (11/09) Page 91 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: AMF JTRS AMF JTRS Program process controls—at each manufacturing site. Consistent with best practices, the sites are expected to demonstrate the ability to produce Technology Maturity production-representative units on pilot lines before DOD certified the AMF JTRS program for entry into beginning low-rate production. Several system development in March 2008 with all five of its manufacturing sites have already demonstrated a critical technologies nearing maturity and capability to produce prototype components or demonstrated in a relevant environment. Prior to the subsystems in a production-relevant environment. start of system development, the AMF JTRS program took steps to develop key product knowledge. In Other Program Issues 2004, the program awarded competitive system AMF JTRS quantities could change depending on the design contracts to two industry teams led by Navy and Marine Corps’ strategy for acquiring Boeing and Lockheed Martin to help mitigate networking capabilities. While all of the services are technical risks and address key integration planning to buy maritime/fixed station radios, the challenges. According to program officials, an Army and Air Force are currently the only services independent technology readiness assessment will planning to purchase the small airborne AMF JTRS be performed in preparation for the small airborne radios. A March 2008 acquisition decision variant production decision, which is scheduled for memorandum removed this requirement for the November 2011. Navy and the Marine Corps and indicated that they plan to rely on the less capable ARC-210 radios for Design Maturity their airborne communications needs. While the The AMF JTRS design appears stable. The program ARC-210 radio is being upgraded, it will not have the reported that all of its expected design drawings waveforms for air-to-air and air-to-ground data were releasable when it completed its design review networking. In August 2008, the Under Secretary of in November 2009. AMF JTRS’ ability to Defense for Acquisition, Technology and Logistics demonstrate that the system meets its performance directed the JTRS joint program executive office, requirements is dependent on waveforms and the Office of the Assistant Secretary of Defense for network management services from the JTRS Networks and Information Integration (NII), along Network Enterprise Domain program. Of the two with the Joint Staff and military services, to assess open items remaining from the design review, issues and options related to replacing currently program officials consider the ability to route and fielded ARC-210 radios with AMF JTRS capabilities. retransmit between radio channels to be high risk. According to an NII official, this assessment has still More specifically, the program is concerned that a not been initiated. needed waveform may not be available in time to allow operational testers to complete testing before Program Office Comments the program’s small airborne variant production In commenting on a draft of this assessment, the decision in November 2011. Program officials Joint Program Executive Office JTRS provided assessed the other open item—security certification technical comments, which were incorporated as from the National Security Agency—as a medium appropriate. risk. Both the program office and National Security Agency agree that there are currently no certification issues with the design. Once the National Security Agency certifies AMF JTRS, any changes will require an additional certification. Certification requirements may impact the system verification testing schedule.

Production Maturity The AMF JTRS program expects to have mature production processes before beginning production. A joint government-contractor assessment team has conducted manufacturing readiness level assessments—which include assessing statistical

Page 92 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JTRS GMR Joint Tactical Radio System (JTRS) Ground Mobile Radios (GMR) DOD’s JTRS program is developing software-defined radios that will interoperate with selected radios and increase communications and networking capabilities. The JTRS GMR program is developing radios for ground vehicles. JTRS GMR depends on waveforms being developed by the JTRS Network Enterprise Domain program, and shares interdependencies with the JTRS Handheld, Manpack, Small Form Fit program as well as the JTRS Airborne and Maritime/Fixed Station program.

Source: Department of Defense.

Concept System development Production

Program Development Design New acquisition Program rebaseline GAO Production Initial start start review baseline directed review decision capability (9/97) (6/02) (12/07) (1/08) (9/08) (11/10) (10/11) (01/13)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Boeing As of Latest Percent Program office: San Diego, CA 06/2002 01/2011 change Funding needed to complete: Research and development cost $1,004.8 $1,697.3 68.9 R&D: $128.3 million Procurement cost $16,159.9 $14,170.4 -12.3 Procurement: $14,170.4 million Total program cost $17,164.7 $15,867.7 -7.6 Total funding: $14,298.7 million Program unit cost $.158 $.182 15.1 Procurement quantity: 86,948 Total quantities 108,388 87,079 -19.7 Acquisition cycle time (months) 55 127 130.9

The JTRS GMR program expects to have its Attainment of Product Knowledge critical technologies mature, design stable, and Production,288 most of its production processes in control by its design, and technology planned October 2011 production decision. maturity e However, the JTRS GMR limited user test and Projection production decision may be delayed to allow the program to test the GMR radio with its final Design and technolo192gy software build and better assess the maturity of maturity the wideband networking waveform. Even if the

JTRS GMR limited user test and production Desired level of knowledg decision are delayed, the Army’s Early Infantry Brigade Combat Team program still plans to Technology request approval to procure the radios for its next maturity 96 two brigades. The JTRS GMR program has yet to fully test key networking capabilities and receive its final National Security Agency certification. The program expects the Office of the Director,

Cost Analysis and Program Evaluation, to 0 Development DOD GAO Production complete a new independent cost estimate in start design review decision January 2011. (6/02) review (11/10) (10/11) (12/07) Page 93 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JTRS GMR

JTRS GMR Program Design Maturity The design of the JTRS GMR appears stable with over 90 percent of the total expected design Technology Maturity releasable to manufacturing. However, until all its The JTRS GMR program started system technologies are mature, key waveforms have been development in 2002 with none of its 20 critical fully integrated and tested, and the program’s final technologies mature and demonstrated in a realistic security certification is received, the potential for environment. The JTRS GMR program expects to design changes remains. have its critical technologies mature by its planned October 2011 production decision. According to the Production Maturity program office, 11 of the 19 current critical The JTRS GMR program has reported that 27 of its technologies are now mature, 7 are nearing maturity, 35 critical manufacturing processes will be in and 1 is still immature. The immature critical statistical control by the program’s planned October technology—bridging/retransmission software—is 2011 production decision. By not having all these to be tested as part of GMR’s multiservice processes in statistical control at production start, operational test and evaluation, which is scheduled there is a greater risk that the radio will not be to begin in the fourth quarter of fiscal year 2012. produced within cost, schedule, and quality targets. However, prior to its production decision, the JTRS GMR relies on the wideband networking program will demonstrate its critical manufacturing waveform—among other waveforms—to meet the processes on a pilot production line. In addition, the requirements of key users, most notably the Early program has delivered 91 engineering development Infantry Brigade Combat Team (E-IBCT) program. model sets for use in developmental and operational While program officials reported the wideband testing. networking waveform to be approaching maturity, the Office of the Director, Defense Research and Other Program Issues Engineering (DDR&E), assessed the waveform’s maturity to be substantially lower in a March 2010 Until a complete and comprehensive cost estimate is technology readiness assessment for the E-IBCT developed, JTRS GMR program costs will remain program. According to the program office, there uncertain. In August 2008, the Under Secretary of have been discussions between the JTRS program Defense for Acquisition, Technology and Logistics executive office and the Army about delaying GMR’s directed the JTRS GMR program to update its cost limited user test—which was scheduled for estimate and revise its acquisition program baseline. completion in December 2010—until later in fiscal Program officials expect the Office of the Director, year 2011. The delay would allow the program to test Cost Analysis and Program Evaluation, to complete the GMR radio with its final software build and an independent cost estimate by August 2011. collect more data for DDR&E to better assess the maturity of the wideband networking waveform. Program Office Comments Testing the radio with its final software build could In commenting on a draft of this assessment, the reduce the risk of late, costly design changes in JTRS Joint Program Executive Office provided production. technical comments, which were incorporated as appropriate. According to program officials, the most significant technical challenge remaining for GMR is meeting security requirements. The program’s security verification test was scheduled for the fourth quarter of fiscal year 2010. The program expects to receive its final security certification from the National Security Agency in the third quarter of fiscal year 2011.

Page 94 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JTRS HMS Joint Tactical Radio System (JTRS) Handheld, Manpack, and Small Form Fit (HMS) DOD’s JTRS program is developing software-defined radios that will interoperate with existing radios and increase communications and networking capabilities. The JTRS HMS program has two concurrent phases of development. Phase 1 includes the Rifleman radio and two small form fit radios. Phase 2 consists of the manpack radio and two additional small form fit radios, all of which are for use in a classified security domain. We assessed phase 1 and made observations on phase 2.

Source: © 2009-2010 General Dynamics.

Concept System development Production

Program/ Design review Design review GAO Low-rate Low-rate development start phase I phase II review decision— decision— Manpack Rifleman radio (4/04) (3/08) (9/09) (11/10) (2/11) (4/11)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: General Dynamics As of Latest Percent C4 Systems, Inc. 05/2004 07/2010 change Program office: San Diego, CA Research and development cost $536.6 $896.2 67.0 Funding needed to complete: Procurement cost $9,352.6 $3,889.9 -58.4 R&D: $79.3 million Total program cost $9,889.2 $4,786.2 -51.6 Procurement: $3,889.9 million Program unit cost $.030 $.022 -26.3 Total funding: $3,969.2 million Total quantities 328,674 215,961 -34.3 Procurement quantity: 215,551 Acquisition cycle time (months) 85 104 22.4

The Rifleman radio’s production decision has been Attainment of Product Knowledge delayed from August 2010 to approximately April Production,288 2011, and the manpack radio’s production design, and technology decision is also at risk. In addition, since August maturity 2009, the program’s estimated procurement cost e increased from $2.5 billion to $3.9 billion, as engineering design models were produced and the Design and technolo192gy program learned more about actual costs. While maturity this amount is less than half the program’s original

estimate, it is planning to buy far fewer radios—in Desired level of knowledg particular the more expensive handheld and manpack radios—than initially planned. Technology maturity 96

0 Development DOD GAO Production start design review decision (4/04) review (11/10) (4/11) (3/08) Page 95 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JTRS HMS

JTRS HMS Program Other Program Issues The production decision for the Rifleman radio has been further delayed from August 2010 to Technology Maturity approximately April 2011, and the manpack radio’s According to the JTRS HMS program, its phase 1 planned February 2011 production decision is also at critical technologies—logical partitioning and risk. The waveforms targeted for the radio’s limited software power management—are nearing maturity. user test are being operationally tested in February In October 2010, the Army assessed the Early 2011; the National Security Agency is not scheduled Infantry Brigade Combat Team and concluded that to complete certification of the manpack radio until one of the JTRS HMS small form fit radios was after successful verification testing in May 2011; and mature, and in January 2011, the Director, Defense the MUOS waveform that the manpack radio is Research and Engineering, concurred with this required to use is not scheduled to be operationally assessment. Additionally, the program office tested until 2012. reported that it will demonstrate the Rifleman radio is fully mature during operational testing in January Since August 2009, the program’s estimated 2011. procurement cost has increased from $2.5 billion to $3.9 billion, as engineering design models were The Army has not assessed the maturity of any of the produced and the program learned more about program’s four phase 2 technologies, but the actual costs. While this amount is less than half the program office reported that it will demonstrate that program’s original estimate, it is planning to buy far the manpack radio is fully mature during operational fewer radios—in particular the more expensive testing in February 2011. The program office has handheld and manpack radios—than initially also reported that the manpack radio is currently planned. meeting its size, weight, and power requirements. Program Office Comments Design Maturity In commenting on a draft of this assessment, the According to the JTRS HMS program office, the JTRS Joint Program Executive Office provided phase 1 design is now stable. The phase 1 Rifleman technical comments, which were incorporated as radio has been reconfigured to address issues appropriate. identified in its 2009 limited user test. The program office reported that the radio now has fewer parts; meets size, weight, and battery requirements; and provides increased reliability and range. The phase 2 design continues to change. JTRS HMS and the Nett Warrior program, which will use the phase 2 small form fit B radio, are investigating alternatives to better accommodate Nett Warrior’s updated requirements, but the program office does not expect this redesign to be a challenge because it will not involve new technology.

Production Maturity According to the JTRS HMS program, its production processes are mature. In 2010, the program identified one critical manufacturing process and reported it was in control. In 2009, the program identified 24 critical manufacturing processes, but it no longer considers any of these processes critical because their maturity has increased.

Page 96 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: LCS Littoral Combat Ship (LCS) The Navy’s LCS is designed to perform mine countermeasures, antisubmarine warfare, and surface warfare missions. It consists of the ship itself, or seaframe, and the mission package it deploys. The Navy is procuring the first four seaframes in two unique designs. The first seaframe (LCS 1) was delivered in September 2008. The second seaframe (LCS 2) followed in December 2009. We assessed both seaframes. See pages 99-100 for an assessment of LCS mission packages.

Sources: Lockheed Martin (left); General Dynamics (right).

Concept System development Production

Program Development Production Production First Second GAO Initial start start decision— decision— ship ship review capability 1st design 2nd design delivery delivery (9/02) (6/04) (12/04) (10/05) (9/08) (12/09) (11/10) (7/12)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: General Dynamics, As of Percent Lockheed Martin 05/2004 Latest change Program office: Washington, DC Research and development cost $873.9 TBD NA Funding needed to complete: Procurement cost $464.6 TBD NA R&D: TBD Total program cost $1,338.5 TBD NA Procurement: TBD Program unit cost $334.622 TBD NA Total funding: TBD Total quantities 4 TBD NA Procurement quantity: TBD Acquisition cycle time (months) 41 98 139.0 Baseline estimates above are for seaframe-related costs only. Research and development funding includes detail design and construction of two ships.

The Navy is building the third and fourth LCS Attainment of Product Knowledge seaframes without having matured all the critical Production,288 technologies or having achieved a stable design. design, and technology Three of 19 seaframe critical technologies are still maturity only nearing maturity and the Navy reported last e year that LCS 3 and LCS 4 began fabrication with only 69 percent and 57 percent of basic and Design and technolo192gy functional drawings complete, respectively. In maturity addition, the Navy’s efforts to resolve technical issues affecting the lead ships have led to design Desired level of knowledg changes to LCS 3 and LCS 4 during construction, several of which remain in progress. Following Technology failed contract negotiations in 2009 for fiscal year maturity 96 2010–funded ships, the Navy twice restructured the program’s acquisition strategy. This process culminated in December 2010 when the Navy awarded contracts for 10 ships of each design between fiscal years 2010 and 2015. 0 Contract Lead ship GAO award fabrication review (12/04) (2/05) (11/10)

Page 97 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: LCS LCS Program The Navy used a concurrent design-build strategy for LCS 1 and LCS 2 seaframes, which proved unsuccessful. Implementation of new design Technology Maturity guidelines, delays in major equipment deliveries, and Sixteen of 19 critical technologies for both LCS strong focus on achieving schedule and performance designs are mature. Three technologies—LCS 1’s goals resulted in increased construction costs. The overhead launch and retrieval system and LCS 2’s Navy’s ongoing efforts to resolve technical issues trimaran hull and aluminum structure—are nearing affecting LCS 1 and LCS 2, implement cost reduction maturity. Further, launch, handling, and recovery measures, and increase mission capability have led systems, which are essential to the LCS to design changes for LCS 3 and LCS 4. These antisubmarine warfare and mine countermeasures changes are significant and have affected the missions, are still being refined for both designs. For configuration of several major ship systems LCS 1, Navy simulations have identified risks in including propulsion, communications, electrical, safely launching and recovering mission systems and navigation. that experience pendulous motion during handling—such as the remote multimission vehicle Other Program Issues and unmanned surface vehicle systems. These After unsuccessful contract negotiations for fiscal operations may be complicated by unacceptably year 2010–funded seaframes, the Navy outlined a high water levels intruding into the ship’s launch bay new acquisition strategy for the LCS program in during high sea states. On LCS 2, the twin boom September 2009 aimed at improving affordability by extensible crane system—designed to launch, selecting one seaframe design for the fiscal year handle, and recover watercraft—contains unproven 2010 ships and beyond. In November 2010, the Navy elements. The Navy reports recent progress on these amended this strategy and proposed contracting for systems including (1) successful operation and 10 ships of each seaframe design through fiscal year movement of an embarked 11-meter rigid-hull 2015. In December 2010, Congress approved this inflatable boat onboard LCS 1 in March 2010,  revised strategy, and the Navy subsequently awarded (2) synthetic lift lines on LCS 2 successfully fixed-price incentive contracts for up to 10 ships completing a 200 percent lift test, and (3) routine each to Lockheed Martin and Austal USA. usage of a straddle carrier to move an 11-meter rigid- hull inflatable boat (with stowage cradle) and Program Office Comments berthing modules around the LCS 2 mission bay. According to the Navy, two industry teams (1) have Navy officials also report that testing of LCS 2’s twin- each designed, built, and delivered to the Navy a boom extensible crane is progressing. lead ship meeting the LCS performance requirements and (2) are currently building their Design and Production Maturity second ships, with lessons learned from the lead The Navy provided historical data on design ships incorporated into the designs. The Navy states completeness that was inconsistent with data it that both designs are stable, with LCS 3 and LCS 4 provided to GAO last year, but officials did not having experienced minimal design changes to-date, respond to requests for clarification. The data and cites impressive learning and investment by provided by the Navy last year indicated that the both shipbuilders as well as significant improvement LCS 3 and LCS 4 began fabrication with only 69 in cost and schedule performance. According to the percent and 57 percent of basic and functional Navy, LCS 3 launched on December 4, 2010, at over drawings complete, respectively. The Navy also 80 percent complete. This level of completeness at could not provide this data for the LCS 1 and LCS 2. launch, and the improvement in cost and schedule GAO’s work on shipbuilding best practices has performance by both shipbuilders, provides the found that leading commercial firms assess a ship Navy confidence that risk of design change and out- design as stable when 100 percent of these drawings of-sequence work is minimal. The Navy also are complete. By delaying construction start until provided technical comments, which were basic and functional design is completed and a incorporated as appropriate. stable design is achieved, shipbuilders minimize the risk of design changes and the subsequent costly rework and out-of-sequence work these changes can drive.

Page 98 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: LCS Modules Littoral Combat Ship-Mission Modules The Navy’s Littoral Combat Ship (LCS) will perform mine countermeasures (MCM), surface warfare (SUW), and antisubmarine warfare (ASW) missions using modular mission packages. Packages include weapons and sensors that operate from MH-60 helicopters or unmanned underwater, aerial, or surface vehicles. Initial packages include engineering development models and production- representative systems of some, but not all, systems planned. Mission capability improves with each package delivered until it reaches a baseline capability.

Source: © Northrop Grumman Corporation.

Concept System development

LCS First First First GAO Milestone Initial Initial program MCM SUW ASW review B–LCS capability capability start delivery delivery delivery MCM SUW (5/04) (9/07) (7/08) (9/08) (11/10) (2QFY11) (2013) (2013)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Northrop Grumman As of Percent Corporation, Integrated Systems 08/2007 Latest change Program office: Washington, DC Research and development cost $484.5 TBD NA Funding needed to complete: Procurement cost $3,211.0 TBD NA R&D: TBD Total program cost $3,695.6 TBD NA Procurement: TBD Program unit cost $57.743 TBD NA Total funding: TBD Total quantities 64 TBD NA Procurement quantity: TBD Acquisition cycle time (months) NA NA NA

The Navy has accepted delivery of five partially Attainment of Product Knowledge capable mission packages. At full baseline Production,288 capability, packages require a total of 21 critical design, and technology technologies, including 11 sensors, 6 vehicles, and maturity 4 weapons for their operation. Most of these e technologies are mature; however, some mission systems have experienced test failures and have Design and technolo192gy not demonstrated the promised capability. maturity Individual systems in the mine countermeasures packages do not meet reliability requirements, and Desired level of knowledg the Navy is currently evaluating alternatives to replace the cancelled Non-Line-of-Sight Launch Technology System (NLOS-LS) and missiles. The Navy is also maturity 96 reexamining the content of the ASW package. Due to developmental delays with key mission systems, the Navy risks acquiring significant numbers of seaframes and mission packages Not Not Not assessed assessed assessed before the mission packages are proven. 0 Development DOD Production GAO start design start review (NA) review (NA) (11/10) (NA) Page 99 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: LCS Modules LCS Modules Program LCS 1 combat system and demonstrated at sea in April 2010. In May 2010, DOD cancelled the Non- Line-of-Sight Launch System due to cost and Technology Maturity technical challenges. Officials note the Navy is At its full baseline capability, operation of the MCM, evaluating other alternatives and expects to SUW, and ASW packages on LCS requires a total of complete evaluation by the second quarter of fiscal 21 critical technologies, including 11 sensors, 6 year 2011. vehicles, and 4 weapons. Of these technologies, 18 are mature and have been demonstrated in a The Navy accepted delivery of one partially capable realistic environment. ASW mission package in September 2008. However, program officials stated that the Navy plans to The Navy has accepted delivery of two partially introduce new mission systems and classified capable MCM mission packages. According to capabilities before procuring additional ASW program officials, in 2010 the MCM mission package packages. Program officials report that the Navy has completed end-to-end testing, and two MCM completed development and testing of the first ASW systems—the AN/AQS-20A sonar and Airborne Laser mission package to evaluate operational concepts Mine Detection System—have completed and refine requirements. developmental testing in separate test events. Two other systems—the Unmanned Surface Vehicle Other Program Issues (USV) and Unmanned Surface Sweep System—have The Navy plans to purchase 18 ships and 13 mission not yet been demonstrated in a realistic packages between fiscal years 2011 and 2015, but environment, and a third—the Remote Minehunting developmental delays in key mission package System (RMS)—has been delayed because of poor systems mean the Navy will acquire significant reliability. Program officials report that the Navy is numbers of seaframes before mission packages are assessing alternative USV designs because the proven. GAO has reported since 2007 on challenges current system does not meet power output developing systems constituting LCS mission requirements necessary to support the towed packages and integrating them with their host surface sweep system. The RMS, which is its own platforms. These challenges have delayed the major defense acquisition program, experienced a planned delivery of baseline capability by several Nunn-McCurdy unit cost breach of the critical years. Until mission package performance is proven, threshold in December 2009, due to cost increases the Navy risks investing in a fleet of ships that does resulting from a 51 percent reduction in quantity and not deliver its promised capability and is largely efforts to improve reliability. In June 2010, the Office constrained to self-defense as opposed to mission- of the Secretary of Defense completed its review and related tasks. certified RMS for continuation. According to Director, Operational Test and Evaluation, officials, Program Office Comments RMS reliability has improved from 7.9 hours to The Navy stated that recent testing has been nearly 45 hours between failures. According to comprehensive, operationally relevant, and program officials, the Navy plans to recommence successful. According to the Navy, the SUW mission RMS production in fiscal year 2015. Further, package supported early deployment of LCS 1, program officials report that the Rapid Airborne providing a counter–illicit trafficking capability. Mine Clearance System has been removed from the Further, the Navy stated that from program package while the Navy evaluates more cost- inception, the acquisition strategy for mission effective alternatives for meeting desired capability package has employed an incremental approach and delivery time frames. remained stable, fielding systems as they achieve the required level of maturity. According to the Navy, The Navy has accepted delivery of two partially those few systems experiencing issues (NLOS-LS capable SUW mission packages and expects to and RMS) are either being replaced with alternative accept delivery of a third mission package in fiscal systems or are targets of increased focus and year 2011. The Navy will resume procuring SUW attention. According to the Navy, the results have packages in fiscal year 2012. The 30 millimeter gun been positive in all cases. In addition, the Navy was test-fired from LCS 1 in September 2009 and provided technical comments, which were according to program officials, integrated with the incorporated as appropriate.

Page 100 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: LHA 6 LHA Replacement Amphibious Assault Ship The Navy’s LHA 6 will replace the LHA 1 Tarawa- class amphibious assault ships. The LHA 6 is a modified variant of the fielded LHD 8 amphibious assault ship and will feature enhanced aviation capabilities and is designed to support all Marine aviation assets in the Expeditionary Strike Group. LHA 6 construction began in December 2008. It is currently scheduled to be delivered in April 2013. The LHA 6 ship class includes three ships. We assessed LHA 6 and made observations on LHA 7 and LHA 8.

Source: U.S. Navy.

Concept System development Production

Program Contract Construction GAO Ship Initial start award start review delivery capability (7/01) (6/07) (12/08) (11/10) (4/13) (10/14)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Northrop Grumman As of Latest Percent Shipbuilding 01/2006 12/2009 change Program office: Washington, DC Research and development cost $217.6 $286.0 31.4 Funding needed to complete: Procurement cost $2,915.4 $6,100.0 109.2 R&D: $48.8 million Total program cost $3,133.0 $6,387.3 103.9 Procurement: $2,866.3 million Program unit cost $3,133.034 $3,193.635 1.9 Total funding: $2,916.2 million Total quantities 1 2 100.0 Procurement quantity: 1 Acquisition cycle time (months) 146 159 8.9 Research and development costs include the LHA 6, LHA 7, and LHA 8. Procurement costs only include the LHA 6 and LHA 7.

The LHA 6 began construction in December 2008 Attainment of Product Knowledge with mature technologies, but a design that was Production,288 only 65 percent complete. Almost all detailed design, and technology design drawings have now been released. In July maturity 2009, the Secretary of the Navy certified that the e LHA 6 program was ready to commence full shipbuilding construction activities. As of Design and technolo192gy September 2010, the program office reported it maturity had conducted unit readiness reviews for all of the

ship’s 216 assembly units, and the shipbuilder had Desired level of knowledg started fabrication on 215 units. The LHA 6 program may incur cost growth due to the need Technology for postdelivery rework of the ship’s deck to cope maturity 96 with the intense, hot downwash from the Joint Strike Fighter.

0 Contract Lead ship GAO award fabrication review (6/07) (12/08) (11/10)

Page 101 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: LHA 6 LHA 6 Program conducted unit-level readiness reviews for all of the ship’s 216 assembly units and the shipbuilder had started fabrication on 215 units. Technology Maturity All LHA critical technologies were mature by the Other Program Issues time the program awarded its construction contract The LHA 6 is likely to experience further cost in June 2007. DOD and the Navy concluded in 2005 growth. Costly postdelivery rework of the ship’s that all LHA 6 components and technologies were deck may be necessary to cope with the downwash fully mature and will have been installed on other from the Joint Strike Fighter. The heat from these ships prior to LHA 6 delivery. Although not aircraft could warp the LHA 6 deck or damage deck considered critical technologies, the program has equipment. The Navy will conduct at-sea testing on identified six key subsystems needed to achieve the USS WASP to determine if and how the LHA 6 and LHA 6’s full capabilities. Five of these are mature, other Joint Strike Fighter–capable ships will need to installed on numerous Navy ships, and do not modify their flight decks. The program office does require modification for the LHA 6. The sixth, the not expect the Navy to finalize a solution to this Joint Precision Approach and Landing System, a issue prior to LHA 6 ship delivery. Global Positioning System-based aircraft landing system, is still in development. While this system is Program Office Comments necessary to realize the LHA 6’s full capabilities, it is In commenting on a draft of this assessment, the not required to meet its operational requirements. Navy stated that the program manager is continually The program office has also previously identified the monitoring shipyard performance and is working machinery control system as a potential risk. The closely with the shipbuilder to identify mitigation shipbuilder expected to commence integrated strategies. The Navy also provided technical testing of the machinery control system for LHA 6 in comments, which were incorporated as appropriate. January 2011 in the land based test equipment.

Design Stability The LHA 6 began construction in December 2008 with only 65 percent of its design complete. Almost all detailed design drawings have now been released. The LHA 7 design will be very close to the LHA 6. Design changes will be limited. These changes include a new firefighting system and radar and command, control, communications, computers, and intelligence updates. Design changes may be more significant on the LHA 8 if the Navy includes a well deck on the ship. All LHA ships except LHA 6 and LHA 7 have a well deck. Officials report that reintroducing the well deck would affect aviation capabilities such as fuel storage space. The Navy will determine the final configuration, capabilities, and cost for the LHA 8 after trade studies are completed in fiscal year 2011. Program officials reported that decisions on the LHA 8 design and the potential increase in funding needed to execute them have not yet been determined.

Production Maturity In July 2009, the Secretary of the Navy certified that the LHA 6 program was ready to commence full shipbuilding construction activities. As of September 2010, the program office reported it had

Page 102 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: MPF(F)/MLP Maritime Prepositioning Force (Future) / Mobile Landing Platform The Navy’s Mobile Landing Platform (MLP) is one of four classes of ships in the Maritime Propositioning Force (Future)—MPF(F)—squadron that supports seabasing. The MLP is designed to facilitate at-sea vehicle and cargo transfer in low-threat environments to support operations ashore. In 2010, the Navy restructured the MPF(F) program, which includes a lower-cost variant of the MLP based largely on a commercial oil tanker. The Navy plans to award the construction contract for the first of three MLP vessels in early 2011.

Source: Computer Sciences Corp.

Concept System development Production

Program GAO Design/ Lead-ship Lead-ship Initial start review construction fabrication delivery operating approval start capability (6/08) (11/10) (3/11) (7/11) (9/13) (3/15)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: General Latest Percent Dynamics/NASSCO As of 07/2010 change Program office: Washington, DC Research and development cost NA $93.7 NA Funding needed to complete: Procurement cost NA $1,426.1 NA R&D: $24.2 million Total program cost NA $1,519.9 NA Procurement: $1,307.0 million Program unit cost NA $506.622 NA Total funding: $1,331.2 million Total quantities NA 3 NA Procurement quantity: 3 Acquisition cycle time (months) NA 81 NA

The MLP program will award its detailed design Attainment of Product Knowledge and construction contract with three of its four Production,288 current critical technologies mature. The design, and technology remaining technology, operations with MLP and maturity its supporting vessels, is nearing maturity. It is not e expected to be mature before construction begins because it requires a complete or near complete Design and technolo192gy MLP to be tested at-sea. The program is currently maturity testing the technology using small-scale models.

As part of the MPF(F) restructuring, the MLP Desired level of knowledg program replaced most of its critical technologies. The redesigned MLP is largely based on a Technology commercial oil tanker. The new design offers less maturity 96 capability, but reduces the program’s cost and schedule. According to program officials, leveraging the design of a commercial oil tanker will allow them to have a higher level of design Not assessed maturity and a lower level of technological risk 0 GAO Contract Lead ship prior to the start of construction. review award fabrication (11/10) (3/11) (7/11)

Page 103 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: MPF(F)/MLP MPF(F)/MLP Program portion of the ship, which will be modified to store supplies and vehicles, as well as the equipment needed for the landing craft interface. In the future, Technology Maturity the MLP may be able to accommodate float-on The MLP program will award its detailed design and modules to provide additional capabilities. construction contract with three of its current critical technologies mature and one nearing Other Program Issues maturity. As a result of the MPF(F) restructure, the Due to resource constraints, the Navy has MLP adopted a new design and lowered the number restructured MPF squadrons by deferring the of critical technologies from five to four, reducing construction of new Large, Medium Speed Roll- MLP capabilities as well as costs. The technologies on/Roll-off vessels, redesignating two classes of are designed to assist in the transfer of cargo ships out of the MPF(F), and reducing the between the MLP and other ships. The three capabilities and costs of the MLP. Additionally, the technologies that have reached maturity—skin-to- MPF(F) concept of operations has changed from skin vehicle transfer with the Large, Medium Speed assembling cargo on-board the MLP to assembling it Roll-on/Roll-off vessel, vehicle transfer with the onshore. Joint High Speed Vessel (JHSV), and the Landing Craft Air Cushion interface—were tested at-sea Program Office Comments using surrogate platforms. Program officials According to the MLP program, it is working with reported that the vehicle transfer technologies— the Office of Naval Research and the Technology which use ramps to connect MLP to the large cargo Readiness Assessment office to reassess MLP vessel or the JHSV at-sea while in motion—were critical technologies. The program anticipates that tested as recently as March 2010. Vehicle transfers this assessment will state that MLP has no critical with the JHSV are currently limited to operations in technologies. The program has also identified a calm waters. The landing craft interface was tested series of production-readiness criteria in the request at sea in March 2010 by loading landing craft at for proposal for the construction contract, including different speeds and approaches in varying sea having certain American Bureau of Shipbuilding states onto a surrogate MLP. While on the MLP, drawings 100 percent complete, the three landing craft may receive cargo, undergo limited dimensional model by zone 90 percent complete, the maintenance, and refuel. The last technology— model by block 65 percent complete, and work landing craft operations with MLP and larger cargo package kits 5 percent complete. According to the vessel connected—requires the simultaneous program, these criteria and the program’s detailed interaction of two of the other technologies. plan for completing all design artifacts to support Program officials do not expect this technology to production will ensure the design is sufficiently reach maturity before construction as it requires a mature for construction. The Joint Requirements complete or near complete MLP for at-sea testing. Oversight Council has also validated all changes in Program officials said it is currently being tested MLP capabilities. The program office also provided using small scale models. technical comments, which were incorporated as appropriate. Design Maturity The MLP has undergone significant design changes due to the MPF(F) program restructure and budget reductions. The new MLP design will offer less cargo, personnel, and aviation capacity, but at a lower cost. The design is based on the Alaska-class crude oil carrier with modifications that allow the MLP to raise and lower itself into the water so that landing craft can float on and off. Program officials reported that the MLP will leverage approximately 60 percent of the commercial design. They also reported 81 percent of preliminary design drawings are complete and the three-dimensional design is underway. The largest changes will be to the central

Page 104 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: MUOS Mobile User Objective System (MUOS) The Navy’s MUOS, a satellite communication system, is expected to provide a worldwide, multiservice population of mobile and fixed-site terminal users with an increase in narrowband communications capacity and improved availability for small terminals. MUOS will replace the Ultra High Frequency (UHF) Follow-On (UFO) satellite system currently in operation and provide interoperability with legacy terminals. MUOS consists of a network of satellites and an integrated ground network. We assessed both the space and ground segments.

Source: © 2008 Lockheed Martin.

Concept System development Production

Program Development Design Production GAO On-orbit Full start start review decision review capability capability (9/02) (9/04) (3/07) (2/08) (11/10) (3/12) (9/15)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Lockheed Martin As of Latest Percent Space Systems 09/2004 06/2010 change Program office: San Diego, CA Research and development cost $3,593.8 $4,151.8 15.5 Funding needed to complete: Procurement cost $2,990.2 $2,613.3 -12.6 R&D: $875.2 million Total program cost $6,622.0 $6,830.2 3.1 Procurement: $1,561.2 million Program unit cost $1,103.658 $1,138.361 3.1 Total funding: $2,436.5 million Total quantities 6 6 0.0 Procurement quantity: 2 Acquisition cycle time (months) 90 112 24.4 The latest cost data do not reflect the current cost of the program, and a new acquisition program baseline has not yet been approved.

All MUOS critical technologies are mature and all Attainment of Product Knowledge design drawings have been released; however, Production,288 design flaws discovered late in production design, and technology continue to pose cost and schedule risks for the maturity program. After a 2009 review of the program e found that the MUOS schedule was optimistic and its budget was inadequate, the program developed Design and technolo192gy more realistic cost and schedule baselines. The maturity new cost baseline has not yet been approved. The current estimate for the first satellite to begin on- Desired level of knowledg orbit operations is March 2012—24 months later than planned when the program began Technology development. The delivery of MUOS capabilities is maturity 96 time-critical due to the operational failures of two UFO satellites. The MUOS program has taken several steps to address any potential capability gap that could occur prior to the first MUOS satellite beginning on-orbit operations. 0 Development DOD Production GAO start design decision review (9/04) review (2/08) (11/10) (3/07) Page 105 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: MUOS MUOS Program the program, the number of defects has decreased slightly over time as the maturity of the manufacturing process has increased. Technology Maturity According to the program office, all eight MUOS Other Program Issues critical technologies are mature and have been The importance of the first MUOS launch increased demonstrated in at least a realistic environment. due to the unexpected failures of two UFO satellites. Based on the current health of on-orbit satellites, Design Maturity UHF communication capabilities are currently According to the program office, the MUOS design is predicted to provide the required availability level stable and the design flaws discovered late in until the first MUOS satellite begins on-orbit production have largely been resolved. However, operations—currently planned for March 2012. design issues with UHF reflectors continue to pose However, the MUOS program is addressing the cost and schedule risks for the program. Specifically, potential for a capability gap by activating dual the UHF reflectors have been redesigned to mitigate digital receiver unit operations on a UFO satellite, signal interference and structural hardware bonding examining the potential of purchasing or leasing issues. According to the program, the late delivery of UHF satellite communications services on a the UHF reflectors—which are on the program commercial satellite, and exploring the feasibility of critical path for the first MUOS satellite launch—is expanded digital receiver unit operations on the the program’s top challenge. The hinges that connect legacy payloads of the MUOS satellites. the solar panels and booms in the solar array wing assembly are also causing unwanted signal In 2009, a Navy-initiated review of the MUOS interference. program found that while it was technically sound, its schedule was optimistic and its budget was According to the program, it has mitigated the inadequate. As a result, the program developed new schedule effects of these design issues by cost and schedule baselines. The acquisition proceeding in September 2010 with system-level program baseline has been under revision since vibration testing, which approximates the level of December 2009, but has not yet been approved. The vibration experienced during launch, prior to prime contract cost baseline for the MUOS program incorporating all of the planned designed was renegotiated in February 2010. According to the modifications for the reflectors and solar panels. program, the prime contract cost baseline, which According to DOD, system-level vibration testing has includes $162 million in engineering change been completed and the risk associated with the proposals, has increased about 61 percent since nonflight compnents are being mitigated by contract award in September 2004. conducting component-level vibration testing on these parts prior to their reinstallation on the Program Office Comments spacecraft. According to the program, the reflectors In commenting on a draft of this assessment, the and solar panels are going through rework and test Navy provided technical comments, which were in parallel with system-level thermal vacuum testing incorporated as appropriate. and are to be available for reinstallation on the spacecraft after system-level testing.

Production Maturity According to the program office, the production maturity of the first MUOS satellite is high. We could not assess production maturity because the program does not collect statistical process control data on its critical manufacturing processes. According to the program office, the space segment does collect, track, and analyze data on manufacturing process defects. While manufacturing defects have contributed to cost growth and schedule delays on

Page 106 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: NMT Navy Multiband Terminal (NMT) The Navy’s NMT is the next-generation maritime military satellite communications terminal. It will be installed in existing ships, submarines, and shore sites. NMT is designed to work with the Air Force’s Advanced Extremely High Frequency (AEHF) Satellite system to enhance protected and survivable satellite communications to naval forces. Its multiband capabilities will also enable communications over existing military satellite communication systems, such as Milstar, Wideband Global SATCOM, and the Defense Satellite Communications System.

Source: © 2008 Raytheon Company.

Concept System development Production

Development Design Low-rate GAO Full-rate Initial Full start review decision review decision capability capability (10/03) (5/08) (7/10) (11/10) (1/12) (9/12) (9/15)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Raytheon As of Latest Percent Program office: San Diego, CA 12/2006 10/2010 change Funding needed to complete: Research and development cost $687.0 $660.9 -3.8 R&D: $57.2 million Procurement cost $1,599.8 $1,143.7 -28.5 Procurement: $1,081.3 million Total program cost $2,286.8 $1,804.7 -21.1 Total funding: $1,138.5 million Program unit cost $6.867 $5.936 -13.6 Procurement quantity: 254 Total quantities 333 304 -8.7 Acquisition cycle time (months) 107 107 0.0

The NMT program entered production in July 2010 Attainment of Product Knowledge with mature critical technologies and a stable Production,288 design, but without demonstrating its critical design, and technology manufacturing processes are in statistical maturity control—a key step for ensuring these processes e are repeatable, sustainable, and capable of consistently producing quality parts. The NMT Design and technolo192gy program began to produce production- maturity representative engineering development models in

May 2008. According to the NMT program, it used Desired level of knowledg these models to mature and baseline its manufacturing processes. The program also plans Technology to complete a manufacturing readiness maturity 96 assessment during fiscal year 2011 to support a full-rate production decision in fiscal year 2012. The NMT program is dependent on AEHF satellites to test its full range of capabilities. The first AEHF satellite was launched in August 2010, 0 Development DOD Production GAO but a propulsion issue has delayed it from start design decision review reaching its planned orbit. (10/03) review (7/10) (11/10) (5/08) Page 107 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: NMT NMT Program data rate can be tested with one AEHF satellite, should the Air Force configure it in that fashion. Additional AEHF satellites provide more coverage Technology Maturity and program officials noted that initial operational The NMT program’s two critical technologies—a capability can be achieved with two installed multiband antenna feed and monolithic microwave systems that have successfully completed system integrated circuit power amplifiers for Q-band and operational verification test. In addition, the NMT Ka-band communication frequencies—are mature. program can provide value to the fleet when it is Both of these technologies have been demonstrated fielded by accessing existing satellite in fully capable, production-representative communication systems such as the Defense engineering development models. Satellite Communications System, Milstar, Wideband Global SATCOM, Interim Polar, and UFO satellite Design Maturity constellations. The NMT’s design is stable. The program has released all of its expected design drawings and The NMT program’s software lines of code have placed the design under configuration control. At its significantly increased since development start to May 2008 design review, program officials reported accommodate software communications that about 70 percent of the expected drawings were architecture requirements. Currently, software releasable to manufacturing. integration testing is over 80 percent complete with over 95 percent of the defects resolved. According to Production Maturity NMT program officials, the NMT program is The NMT program office entered production in July containing most of the defects that it finds within 2010 without demonstrating that its manufacturing phase, which is a good indicator because it is more processes were in statistical control—a key step for efficient to correct problems within the phase in ensuring these processes are repeatable, which they occur. sustainable, and capable of consistently producing high-quality parts. During a June 2008 technology Program Office Comments readiness assessment, the program identified three In commenting on a draft of this assessment, the critical manufacturing processes related to the Q- Navy stated that the NMT program has successfully band and Ka-band monolithic microwave integrated entered the production phase and continues to circuits and the Q/Ka radome. The NMT program successfully progress to provide deployed naval began to produce production-representative commanders with assured access to secure, engineering development models in May 2008. protected, command and control and According to the NMT program, it used its communication capabilities to support the exchange production run of 33 engineering development of warfighter-critical information. It will support the models to mature and baseline its manufacturing Navy’s net-centric FORCEnet architecture and act as processes. This will allow the program to begin an enabler for transforming operational capability tracking statistical process control data. A available to the warfighter. The Navy also provided manufacturing readiness level assessment is technical comments, which we incorporated as scheduled to occur during fiscal year 2011 to appropriate. support a full-rate production decision review in fiscal year 2012.

Other Program Issues The NMT program is dependent on AEHF satellites to test its full range of capabilities. The first AEHF satellite was launched in August 2010; however, a faulty satellite propulsion system will delay the satellite from reaching its planned orbit by about 7 to 9 months. Delays with AEHF capability directly affect the ability of the NMT program to test the new higher data rate communications capability. However, NMT officials stated that the new higher

Page 108 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: P-8A P-8A Poseidon The Navy’s P-8A Poseidon is a Boeing 737 commercial derivative that will replace the P-3C Orion. Its primary roles are antisubmarine warfare; antisurface warfare; and intelligence, surveillance, and reconnaissance. The P-8A is a part of a family of systems that share the integrated maritime patrol mission and support the Navy’s maritime warfighting capability. The program plans to field capabilities in three increments. We assessed increment one.

Source: U.S. Navy.

Concept System development Production

Program Development Design Low-rate GAO Full-rate Initial Last start start review decision review decision capability procurement (3/00) (5/04) (6/07) (8/10) (11/10) (4/13) (7/13) (2019)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Boeing As of Latest Percent Program office: Patuxent River, MD 05/2004 11/2010 change Funding needed to complete: Research and development cost $7,420.2 $7,795.0 5.1 R&D: $1,857.7 million Procurement cost $23,020.1 $23,738.8 3.1 Procurement: $21,751.5 million Total program cost $30,575.9 $32,352.6 5.8 Total funding: $24,327.4 million Program unit cost $265.877 $265.186 -0.3 Procurement quantity: 111 Total quantities 115 122 6.1 Acquisition cycle time (months) 160 160 0.0

The P-8A entered production in August 2010 with Attainment of Product Knowledge mature technologies and a stable design. The Production,288 program completed a production readiness review design, and technology in January 2010 and demonstrated its airframe maturity manufacturing processes on a commercial line e prior to the production decision. However, several parts of the P-8A, including the sonobouy Design and technolo192gy launcher, auxiliary fuel tanks, and a new fuel tank maturity safety system, have manufacturing readiness levels that are lower than recommended for the Desired level of knowledg start of production. The airframe on the P-8A program has been designated as a commercial Technology item. The Defense Contract Audit Agency has maturity 96 expressed concern about the designation because of the extent of the modifications being made to the aircraft. In addition, according to the program office, the Defense Contract Management Agency

has cited limited access to commercial production 0 Development DOD Production GAO facilities as a concern. start design decision review (5/04) review (8/10) (11/10) (6/07) Page 109 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: P-8A P-8A Program engineering. In addition, according to the program office, both the Office of the Secretary of Defense and the Defense Contract Management Agency Technology and Design Maturity (DCMA) have expressed concerns about the limited The P-8A entered production in August 2010 with access to production facilities and limited mature technologies and a stable design. An surveillance of aircraft parts afforded by the independent technology readiness assessment of the commercial item designation. program was conducted in December 2009 to support the production decision. The assessment Prior to entering production, an operational identified one current critical technology, the hydro- assessment of the P-8A found that the system carbon sensor, and rated it mature. The sensor has demonstrated the expected level of maturity or been tested in ground-based applications, but has exceeded all test thresholds. The assessment was not been demonstrated in an aircraft. While the ESM conducted in the program’s Weapon System digital receiver was considered a critical technology Integration Laboratory (WSIL) and not with an during development, program officials stated that operationally representative aircraft. The Navy this technology was no longer identified as such operational testers stated that conducting the because it is mature and has been demonstrated on operational assessment in the WSIL proved to be the E/A-18G. However, no formal ESM flight testing useful in determining and evaluating the preliminary has been conducted on the P-8A. According to the risks in the development of the P-8A system, but that program office, another formerly identified critical characterizing system risks based on this data alone technology, the sonobouy launcher, is scheduled to represented a major limitation. According to the begin testing in a realistic environment in fiscal year Navy testers, subsequent flight tests conducted in 2011. June 2010 have been successful with only minor issues observed. Initial operational test and Production Maturity evaluation will begin in 2012. The critical manufacturing processes for the P-8A airframe are proven, but manufacturing readiness Program Office Comments levels are lower than recommended for the start of In commenting on a draft of this assessment, the production. The P-8A program completed a Navy stated that since the P-8A was competitively production readiness review in January 2010 and awarded and more than one offer was received, it demonstrated its critical airframe manufacturing did not ask for certified cost or pricing data for the processes on a commercial line prior to its August system development and demonstration contract 2010 production decision. The airframe is being with Boeing Defense, Space and Security (BDS). procured as a commercial item and has stable The Navy further explained that as the airframe is production processes that support production rates purchased as an interdivisional commercial item, in excess of 32 airframes per month. However, DCMA does not have independent access to inspect several parts of the P-8A, including the sonobouy it in Boeing Commercial Airplanes’ (BCA) facilities. launcher, auxiliary fuel tanks, and new fuel tank However, the DCMA and the Navy may accompany safety system are currently assessed at BDS during BCA selected quality reviews. These manufacturing readiness levels that are lower than events are typical and customary for any customer those recommended for the start of production. of BCA. Inspections apply to the aircraft once it reaches the BDS facilities where DCMA can inspect Other Program Issues any part of the end product. The Navy also provided The P-8A airframe has been designated as a technical comments, which were incorporated as commercial item. As a result, the contractor is not appropriate. required to submit cost or pricing data to the government. According to the Navy, it weighed the assumed cost and benefits before making the commercial item designation. The Defense Contract Audit Agency has expressed concern about the designation because of the extent of the modifications being made to the aircraft, which include an estimated $460 million in nonrecurring

Page 110 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: PATRIOT/MEADS CAP Fire Unit

PATRIOT/Medium Extended Air Defense System (MEADS) Combined Aggregate Program (CAP) Fire Unit

The Army’s PATRIOT/Medium Extended Air Defense System (MEADS) program transitions the PATRIOT missile system to MEADS. MEADS is intended to provide low-to-medium-altitude air and missile defense to counter, defeat, or destroy tactical ballistic missiles, cruise missiles, or other air- breathing threats. MEADS is being developed by the United States, Germany, and Italy. We assessed the MEADS fire unit, which includes launchers, radars, a battle management component, and reloaders. We did not assess the PATRIOT missile.

Source: U.S. Army.

Concept System development Production

Program/ Critical GAO Full-rate Initial development start design review review decision capability (8/04) (8/10) (11/10) (11/12) (9/17)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: MEADS International As of Latest Percent Program office: Huntsville, AL 08/2004 12/2009 change Funding needed to complete: Research and development cost $5,229.5 $4,820.3 -7.8 R&D: $2,820.2 million Procurement cost $13,847.8 $13,693.0 -1.1 Procurement: $13,693.0 million Total program cost $19,077.3 $18,513.3 -3.0 Total funding: $16,513.2 million Program unit cost $397.444 $385.694 -3.0 Procurement quantity: 48 Total quantities 48 48 0.0 Acquisition cycle time (months) 157 157 0.0 The cost, schedule, quantity, and funding data are for the MEADS fire unit.

The MEADS program completed a system-level Attainment of Product Knowledge critical design review in August 2010 with its Production,288 technologies mature and design stable. The design, and technology MEADS member nations held a program review in maturity October 2010, according to officials, to decide e whether or not to continue with the program and whether or not to modify the system to use a Design and technolo192gy unified battle management control system being maturity developed by the Army’s Integrated Air and

Missile Defense program. If the Army and member Desired level of knowledg nations decide to use the new unified battle management control system, the MEADS program Technology will require increased time and funding to maturity 96 develop, field, and integrate this system into the existing fire unit software and hardware. The MEADS program is expected to be rebaselined following the program review if the decision is made to continue it. 0 Development DOD GAO Production start design review decision (8/04) review (11/10) (11/12) (8/10) Page 111 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: PATRIOT/MEADS CAP Fire Unit PATRIOT/MEADS CAP Fire Unit The MEADS program is expected to be rebaselined Program following the program review. MEADS officials expect the program’s design and development phase to be extended by 18 months due in part to issues Technology Maturity with BMC4I and sensor requirements and an All five of the MEADS critical technologies— underestimation of the sensor development effort launcher electronics, multifunction fire control that delayed the program’s critical design review. radar exciter, multifunction fire control radar Program officials stated that the increased cost transmit/receive module, slip ring, and spray cooling associated with the schedule extension is expected system—are mature. to be shared among the three member nations. Details regarding the schedule extension and its Design Maturity effect on the program were not available as The MEADS program completed its system-level negotiations had not begun among the member critical design review in August 2010 and its design nations. According to the program’s Selected is stable. At critical design review the program had Acquisition Report, the MEADS contract was released 93 percent of the total expected design expected to be amended in the first quarter of fiscal drawings across the five major end items. The year 2011 to incorporate any programmatic changes. MEADS battle management, command, control, communications, computer, and intelligence The MEADS program is at risk of not meeting (BMC4I) software and hardware was the only major several technical performance measures, including end item with less than 90 percent of its drawings assembly, disassembly, and emplacement times, released. Only 76 percent of BMC4I drawings were especially in extreme temperatures. According to releasable by the design review because, according officials, the Army has approved a request for relief to program officials, one of the international for several system performance specifications partners came in with a late request to change the related to the transportability of various collapsible roof design to a fixed roof. As of components on C-130 aircraft as well as CH-47 and December 2010, the program has released 98 CH-53 helicopters. The MEADS program faces other percent of the expected drawings for the BMC4I and transportability challenges as well because the 98 percent across the five major end items. vehicles used to move the system do not meet all NATO road requirements. Other Program Issues The MEADS member nations held a program review Program Office Comments in October 2010, according to officials, to decide In commenting on a draft of this assessment, whether or not to continue with the program and if program officials noted that the MEADS program is so, which battle management system to use—the over 6 years into development, that fabrication is current MEADS BMC4I tactical operations center or well underway, and that initial major end-item the Army’s Integrated Air and Missile Defense Battle deliveries would begin in December 2010. They Command System (IBCS). This program decision stated that integration and testing activities are was postponed until December 2010. The Army planned to start during calendar year 2011. While the plans to use IBCS to control and manage sensors United States is still planning to use the IBCS, the and weapons, such as PATRIOT and the Joint Land- international partners are not, and a program Attack Cruise Missile Defense Elevated Netted decision is still anticipated by the end of December Sensor System, and support the engagement of air 2010. Program officials concluded that requirements and missile threats. However, the MEADS BMC4I is satisfaction, software maturity, and cost growth further along in its development than the IBCS, continue to be concerns. The program also provided which entered system development in December technical comments, which were incorporated as 2009. As a result, the MEADS program would require appropriate. increased time and funding to develop, field, and integrate the IBCS into the existing fire unit software and hardware, if the decision is made to use it.

Page 112 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Reaper Reaper Unmanned Aircraft System The Air Force’s MQ-9 Reaper is a multirole, medium- to-high-altitude endurance unmanned aerial vehicle system capable of flying at higher speeds and higher altitudes than its predecessor, the MQ-1 Predator A. The Reaper is designed to provide a ground-attack capability to find, fix, track, target, engage, and assess small ground mobile or fixed targets. Each system consists of four aircraft, a ground control station, and a satellite communications suite. We assessed increment 1, which consists of two configurations, Block 1 and Block 5.

Source: U.S. Air Force.

Concept System development Production

Program Development Block 1 GAO Required Initial Block 5 Block 5 start start low-rate review assets capability low-rate full-rate decision available decision production decision (1/02) (2/04) (2/08) (11/10) (12/10) (3/11) (3/11) (3/13)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: General Atomics As of Latest Percent Aeronautical Systems, Inc 02/2008 06/2010 change Program office: Wright-Patterson AFB, Research and development cost $413.9 $806.2 94.8 OH Procurement cost $2,080.2 $10,171.9 389.0 Funding needed to complete: Total program cost $2,597.9 $11,131.8 328.5 R&D: $448.2 million Program unit cost $24.742 $28.470 15.1 Procurement: $7,980.0 million Total quantities 105 391 272.4 Total funding: $8,534.7 million Acquisition cycle time (months) 79 82 3.8 These costs are for Increment 1 of the Reaper program. Procurement quantity: 288

The Block 1 Reaper is in production with critical Attainment of Product Knowledge technologies that are mature and a design that is Production,288 stable. We did not assess its production maturity. design, and technology The MQ-9 program plans to make numerous maturity enhancements in Block 5, including system power e increases, modernized crew stations, and improvements to the primary data link. The Design and technolo192gy program office judged these improvements to be maturity technologically mature, but they still must be integrated and tested on the MQ-9 system. Total Desired level of knowledg aircraft quantities have increased more than 500 percent since fiscal year 2007 and the program is Technology incorporating several urgent operational needs maturity 96 from the warfighter. Although the Reaper’s initial operational testing was completed in August 2008, full-up testing of two key performance parameters was delayed to November 2012 during Block 5 testing. The Air Force plans to begin development 0 Development DOD Production GAO of Increment 2 in late fiscal year 2012. start design decision review (2/04) review (2/08) (11/10) (2/09) Page 113 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Reaper Reaper Program aircraft quantities have increased by over 500 percent since fiscal year 2007 and the system’s performance requirements have continued to Technology Maturity change. In addition to the Block 5 capability The Reaper’s Block 1 critical technologies are upgrade, the program is also incorporating several mature. The Air Force has identified numerous urgent operational requirements from the technology enhancements for Block 5 that are warfighter, such as data link encryption, wide area / expected to improve the capability of existing high resolution surveillance, and a capability to onboard subsystems and ground control stations. detect dismounted soldiers. Meeting these demands These enhancements include power increases, radar has put a stress on the program’s resources. A recent and ground control station upgrades, a secure data systems engineering review noted that the link, and heavyweight landing gear. The program contractor’s resources have been overburdened by office judged these improvements to be the need to balance software development, support technologically mature, but they still must be ongoing operations, and enhance system capability. successfully integrated and tested on the MQ-9 It also found that the Reaper program lacks system. For example, the encryption of the data sufficient software metrics to allow proper using the primary data link increases the time to developmental resource and schedule planning. transmit data. These transmission delays could result in hard landings, which may damage the The Block 1 Reaper completed initial operational aircraft. The program office is currently evaluating a testing in August 2008. Testers found that it was range of hardware and software solutions to this effective in the killer role, but problems associated problem and plans to test them operationally in with radar and the network prevented them from November 2012. The program plans to undergo a evaluating the hunter and net-ready capability. To Block 5 technology readiness assessment in support enable testers to fully evaluate the hunter capability, of its low-rate production decision by March 2011. the Air Force is upgrading the radar’s ground moving target indicator and target recognition/classification Design Maturity capability, and integrating the radar into the crew According to the program office, the Block 1 Reaper station. Full-up testing of these capabilities was design is stable and all engineering drawings have delayed and will be completed during the Block 5 been released. The MQ-9 program plans to conduct a initial operational testing, scheduled for November formal critical design review on the Block 5 2012. configuration in December 2010. At that time, it expects to know the number of additional drawings The Air Force plans to begin development of needed for this configuration. increment 2 of the MQ-9 Reaper in late fiscal year 2012. This increment will include the small diameter Production Maturity bomb, an automatic take-off and landing capability, We did not assess production maturity because the a deicing system, and national airspace certification. MQ-9 program does not use statistical process controls. The program uses other quality control Program Office Comments measures such as scrap, rework, and repair to track In commenting on a draft of this assessment, the Air product quality. The program is in production and Force provided technical comments, which were contracted for 103 aircraft. The contractor has a incorporated as appropriate. continuous improvement program that includes manufacturing process goals, which are updated as they are met. The Air Force has also conducted several manufacturing reviews of the contractor’s facilities and determined that the production capacity is sufficient to meet the expected demand.

Other Program Issues Since its inception, the MQ-9 program has followed a concurrent development and production strategy in order to respond to urgent operational needs. Total

Page 114 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: SSC Ship to Shore Connector (SSC) The Navy’s Ship to Shore Connector is expected to provide transport of personnel, weapon systems, equipment, and cargo from ships to the shore. SSC is the replacement for the Landing Craft, Air Cushion (LCAC), which is facing the end of its service life. The SSC will deploy in existing and planned Navy well deck amphibious ships and will be used for assault and nonassault operations. It is expected to operate independent of tides, water depth, underwater obstacles, ice, mud, or beach conditions.

Source: U.S. Navy.

Concept System development Production

Program GAO Preliminary Development Contract award— Lead ship Initial start review design review start design/construction fabrication start capability (5/09) (11/10) (3/11) (3Q/FY11) (TBD) (TBD) (TBD)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: TBD Latest Percent Program office: Washington, DC As of 08/2010 change Funding needed to complete: Research and development cost NA $435.7 NA R&D: TBD Procurement cost NA $3,885.8 NA Procurement: TBD Total program cost NA $4,343.7 NA Total funding: TBD Program unit cost NA $59.503 NA Procurement quantity: TBD Total quantities NA 73 NA Acquisition cycle time (months) NA TBD NA

The SSC program plans to award the detail design Attainment of Product Knowledge and construction contract for the lead ship in Production,288 fiscal year 2011 with all five potential critical design, and technology technologies nearing maturity or mature. maturity According to the program office, the SSC will be e the first government-led Navy ship design in 14 years. Aligned with goals from Office of the Under Design and technolo192gy Secretary of Defense for Acquisition, Technology maturity and Logistics’ efficiency initiative, the Navy is focused on balancing costs with capabilities Desired level of knowledg during the technology development phase in order to formulate requirements that are technically Technology achievable within known fiscal constraints. The maturity 96 Under Secretary has also emphasized affordability and encouraged programs to make tradeoffs in order to stay within the established costs for the program. Not Projection assessed 0 GAO Contract Lead ship review award fabrication (11/10) (TBD) (TBD)

Page 115 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: SSC SSC Program development phase. The Navy noted that the program is considered low risk, technically sound, and is postured for events leading up to Technology Maturity development start, program initiation, and release of The SSC program expects all five potential critical a request for proposal for detail design and technologies to be mature or nearing maturity by the construction and entry into the engineering and time the detail design and construction contract for manufacturing development phase. The Navy also the lead ship is awarded. The program has identified provided technical comments that were five critical technologies—the aluminum buoyancy incorporated as appropriate. box, gas turbines, fire suppression system, composite shaft, and composite lift fan. According to the program office, some components of these technologies are already in use in the Navy fleet. For instance, several potential SSC candidate engines are used in the aircraft industry today. According to program officials, there are risks associated with readying them for ships. The aluminum chosen is the same alloy that is in use on the second Littoral Combat Ship, and composites are used throughout the Navy fleet, including on the LCACs. The Navy plans to release the request for proposal for the detail design and construction of the SSC in the first quarter of 2011.

Other Program Issues According to Navy officials, the SSC is the first government-led Navy ship design in 14 years. The Under Secretary of Defense for Acquisition, Technology and Logistics has emphasized affordability and encouraged programs in the technology development phase to make tradeoffs in order to stay within the established costs for the program. Accordingly, the Navy is focusing on producing a design that reduces maintenance costs and balances performance requirements against life- cycle costs. The SSC is expected to have greater lift, a lower fuel consumption rate, and less expected maintenance than the LCAC. The Navy plans to achieve this through a series of design changes, including the extensive use of composite materials, a simpler and more efficient drive train, and more powerful, fuel-efficient engines. In validating the SSC’s key performance parameters in June 2010, the Joint Requirements Oversight Council (JROC) required the program to return to the JROC if costs exceed 10 percent of the approved program baseline.

Program Office Comments In commenting on a draft of this assessment, the Navy stated that the JROC validated the SSC requirements that have defined the SSC technical design parameters within the technology

Page 116 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: SDB II Small Diameter Bomb (SDB), Increment II The Air Force’s Small Diameter Bomb (SDB) Increment II is planned to provide attack capability to moving or stationary mobile targets in adverse weather from standoff range. It combines radar, infrared, and semi-active laser sensors in a multi- mode seeker to acquire, track, and engage targets. It uses a weapons data link from host aircraft as well as GPS and an inertial navigation system to achieve accuracy. SDB II will integrate with the F-15E and the Navy and Marine Corps Joint Strike Fighter, and with other aircraft, such as the F-22A.

Source: © 2010 Raytheon Company.

Concept System development Production

Program Development GAO Critical Low-rate Initial Initial Last start start review design decision capability capability procurement review F-15E F-35B/C (5/06) (7/10) (11/10) (1/11) (1/13) (7/16) (6/18) (9/23)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Raytheon As of Latest Percent Program office: Eglin AFB, FL 10/2010 10/2010 change Funding needed to complete: Research and development cost $1,618.8 $1,618.8 0.0 R&D: $1,071.0 million Procurement cost $3,009.1 $3,009.1 0.0 Procurement: $3,009.1 million Total program cost $4,627.9 $4,627.9 0.0 Total funding: $4,080.1 million Program unit cost $.270 $.270 0.0 Procurement quantity: 17,000 Total quantities 17,163 17,163 0.0 Acquisition cycle time (months) 72 72 0.0

The SDB II program entered system development Attainment of Product Knowledge in July 2010 with all four of its critical Production,288 technologies nearing maturity. In an April 2010 design, and technology technology readiness assessment, each maturity technology was found to need additional e development to demonstrate the required level of maturity for operational use. While some of the Design and technolo192gy SDB II technologies are being utilized in other maturity systems, they are being applied in new ways on

this program. In addition, the integration of those Desired level of knowledg technologies into the constrained SDB II design is a risk for the current development schedule. Technology Further, the program already faces funding maturity 96 shortfalls. When the program was approved to enter development, it was granted a waiver from the requirement to provide full funding. The funding shortfall was estimated to be about 22 Not Not assessed assessed percent of the required funding over the life of the 0 Development GAO DOD Production program. start review design decision (7/10) (11/10) review (1/13) (1/11) Page 117 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: SDB II

SDB II Program Other Program Issues In July 2010, the Under Secretary of Defense for Acquisition, Technology and Logisitcs approved the Technology Maturity SDB II to enter engineering and manufacturing The SDB II program entered system development in development. The program will use a fixed-price July 2010 with all four of its critical technologies incentive contract for this phase of the program. The nearing maturity. In April 2010, an independent Under Secretary also approved an aggressive technology readiness assessment found that the procurement schedule in an effort to promote SDB II data link, payload (warhead), seeker, and affordability and productivity. However, the target classifier had been demonstrated in a relevant acquisition decision memo that outlined these environment. The assessment team based their decisions stated that it will be a challenge to execute conclusions on modeling and simulation, as well as this strategy because of DOD’s track record of captive flight, static and dynamic warhead, and procuring weapons at less than economically other testing methods. Each technology was found beneficial rates due to budget pressures. In addition, to need additional development to demonstrate the the program already faces funding shortfalls. When required level of maturity for operational use. In the program was approved to enter development, it addition, while each of these technologies has been was granted a waiver from the requirement to fielded in one or more weapon systems, they are provide full funding. The program is not currently being applied in new ways on this program and must funded to the Air Force’s cost estimate or its current be integrated into the constrained SDB II design. acquisition program baseline. The shortfall was The data link is a new application of an existing estimated to be about 22 percent of the required technology and will require a tremendous leap funding over the life of the program. forward in packaging. The payload (warhead) will be used in a way on SDB II that is beyond its current Program Office Comments demonstrated capability, and fuze development has The program office offered technical comments, been and continues to be a problem. The seeker which were incorporated as appropriate. combines proven sensor technologies, but in a smaller design. The target classifier is a new technology in a weapon system context. Finally, Joint Strike Fighter integration issues represent a substantial risk for the program and the Navy because the aircraft may not be available for environmental and integration testing until after the SDB II has completed its initial development. The environments of the Joint Strike Fighter may cause a re-design of a portion of the weapon or limitations in the weapon’s employment.

Design Maturity The SDB II program held its critical design review in January 2011. We could not assess SDB II design maturity because data on design drawings were not available. As an entrance criteria for holding the design review, the program had planned to determine if 95 percent of the system’s drawings were completed and under configuration control. Program officials stated that it appeared the design as presented is capable of achieving the desired requirements for the system. If the program’s post– critical design review assessment is successful, the contractor will be cleared to start building hardware for the SDB II program.

Page 118 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: SBIRS High Space Based Infrared System (SBIRS) High Program The Air Force’s SBIRS High satellite system is being developed to replace the Defense Support Program and perform a range of missile warning, missile defense, technical intelligence, and battlespace awareness missions. SBIRS High consists of four satellites in geosynchronous earth orbit (GEO) plus two replenishment satellites, two sensors on host satellites in highly elliptical orbit (HEO) plus two replenishment sensors, and fixed and mobile ground stations. We assessed the space segment and made observations about the ground segment.

Source: © 2007 Lockheed Martin Corporation.

Concept System development Production

Program Development Design review / First Second GAO First Second start start production decision sensor sensor review satellite satellite delivery delivery launch launch (2/95) (10/96) (8/01) (8/04) (9/05) (11/10) (4/11) (4/12)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Lockheed Martin As of Latest Percent Corporation 10/1996 12/2009 change Program office: El Segundo, CA Research and development cost $4,304.5 $10,626.7 146.9 Funding needed to complete: Procurement cost $0.0 $5,065.3 NA R&D: $1,844.4 million Total program cost $4,520.6 $15,938.5 252.6 Procurement: $3,085.6 million Program unit cost $904.111 $2,656.409 193.8 Total funding: $4,954.3 million Total quantities 5 6 20.0 Procurement quantity: 3 Acquisition cycle time (months) 86 TBD NA The 1996 data show no procurement cost as the Air Force planned to use research and development funds to buy all five satellites. The cost of the HEO replenishment sensors is not included in either column.

According to the program office, SBIRS High Attainment of Product Knowledge critical technologies are mature, its design is Production,288 stable, and its manufacturing processes have been design, and technology proven; however, continued difficulties with flight maturity software development could add to the program’s e cost overruns and schedule delays. According to the program office, significant progress has been Design and technolo192gy made on flight software testing. However, various maturity subsystem- and system-level qualification testing

remains and the Defense Contract Management Desired level of knowledg Agency (DCMA) has reported that the effort required to finalize the flight software is likely to Technology further delay the launch date of the first GEO maturity 96 satellite. The program office’s best-case estimate is that the first GEO satellite will launch in April 2011—4 months later than previously estimated and roughly 9 years later than originally planned.

0 Development DOD Design review/ GAO start design production review (10/96) review decision (11/10) (NA) (8/01) Page 119 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: SBIRS High SBIRS High Program Developing the complex flight software subsystem designed to monitor the health and status of the spacecraft has already caused multiple delays, and Technology Maturity DCMA has reported that the remaining software According to the SBIRS High program office, all effort will likely further delay the launch date of the three critical technologies—thermal management, first GEO satellite. According to the program office, onboard processing, and the infrared sensor—are significant progress has been made on flight mature. software testing, but various subsystem- and system- level qualification testing remains. For example, the Design Maturity fault management system, which has caused delays The SBIRS High hardware design appears stable. in the past, was scheduled to undergo testing in late According to the program office, over 99 percent of 2010 to determine whether the system will perform the total expected design drawings are releasable. as expected. According to DCMA, these tests had to Functional testing of the first GEO spacecraft in be completed by the end of 2010 for the program to 2009 revealed solder fractures on some hardware have a realistic chance of launching the first GEO components, which contributed to satellite delivery satellite in April 2011—the program office’s best- delays. The program disassembled and tested these case estimate. components and determined that redesign was unnecessary and they were suitable for use as-is. According to program officials, the development of the SBIRS High ground system is on track, and the Hosted HEO sensors are currently on-orbit, and system will be available to process the data program officials report that they are operational. generated from the first GEO satellite when it The program plans to buy replenishment sensors reaches its orbit. If the first GEO satellite launches that will differ only slightly in design. According to in April 2011, program officials expect that satellite the program office, the design changes will address data will be certified for use in missile warning parts obsolescence and electromagnetic operations by November 2012. interference issues that affected the original sensors. Those interference issues led the program to issue Program Office Comments waivers to accept the original sensors for In commenting on a draft of this assessment, the operational use, even though they did not meet all program office stated that the first GEO satellite the program’s specifications. Replenishment sensors successfully completed all integration testing. In are scheduled for delivery to the host for integration mid-December 2010, the final system-level test was in fiscal years 2012 and 2015. completed, and installation began of components, such as solar array wing assemblies and a Production Maturity deployable light shade. Flight software run-for- According to the program office, the manufacturing record activities for the first GEO satellite are processes for SBIRS High are proven since the first expected to be completed in February, supporting and second GEO satellites and the first two HEO the start of launch processing in March. Independent sensors have been built. review teams verified that an April 2011 launch is achievable. Ground software required for launch has Other Program Issues been verified. The follow-on production contract The estimated cost of the program continues to procures the third and fourth GEO satellites grow, and its schedule is at risk for further delays. completing the original SBIRS constellation, plus DCMA projects nearly $600 million in cost overruns two HEO replenishment payloads; it was definitized at contract completion, more than twice the amount in June 2010, and values roughly $3 billion. The reported last year. Additional contract cost increases program office also provided technical comments, and schedule delays are expected due in part to the which were incorporated as appropriate. continued underestimation of the effort required to finalize and test the flight software. The program office is working to rebaseline the SBIRS High contract cost and schedule estimates for the sixth time; it will then revise its acquisition program baseline to more realistic cost and schedule goals.

Page 120 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: SM-6 Standard Missile-6 (SM-6) Extended Range Active Missile (ERAM) The Navy’s Standard Missile-6 (SM-6) is a surface-to- air missile launched from Aegis destroyers and cruisers to provide ship self-defense, fleet area defense, and theater air defense. Combining legacy Standard Missile (SM) and Advanced Medium-Range Air-to-Air Missile (AMRAAM) hardware and technology, SM-6 will allow for over-the-horizon engagement, improved capability at extended ranges, and capability to receive in-flight updates from Aegis ships. SM-6 Block 1 is in production. Follow-on blocks will be developed to meet future threats.

Source: Raytheon Missile Systems.

Concept System development Production

Program Design Low-rate GAO Initial Full-rate start review decision review capability decision (6/04) (3/06) (8/09) (11/10) (9/11) (12/11)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Raytheon Missile As of Latest Percent Systems 07/2004 07/2010 change Program office: Arlington, VA Research and development cost $1,057.9 $972.0 -8.1 Funding needed to complete: Procurement cost $4,558.1 $5,160.9 13.2 R&D: $72.4 million Total program cost $5,616.0 $6,132.8 9.2 Procurement: $4,940.0 million Program unit cost $4.680 $5.111 9.2 Total funding: $5,012.3 million Total quantities 1,200 1,200 0.0 Procurement quantity: 1,170 Acquisition cycle time (months) 75 87 16.0

The SM-6 program’s concurrent testing and Attainment of Product Knowledge production strategy puts the program at increased Production,288 risk of cost growth and schedule delays. The design, and technology program is in low-rate production with 30 missiles maturity under contract and deliveries expected to begin in e March 2011. However, the program has not completed developmental testing to prove that the Design and technolo192gy design meets performance and reliability maturity requirements, and the risk of design changes

remains. Recent test failures have increased the Desired level of knowledg risk that unexpected design changes could result in costly rework for the 30 missiles already under Technology contract and schedule delays. To receive approval maturity 96 to enter full-rate production, the program must successfully complete flight testing, demonstrate reliability, and achieve production maturity. Full testing of SM-6 capabilities will not occur until

after full-rate production is well underway. 0 Development DOD Production GAO start design decision review (6/04) review (8/09) (11/10) (3/06) Page 121 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: SM-6 SM-6 Program testing and obtain approval prior to awarding subsequent contracts. Despite the test failures that led to the suspension of at-sea developmental testing Technology and Design Maturity in May 2010, the Under Secretary approved low-rate According to the program office, all SM-6 critical production of an additional 11 missiles and the technologies were mature and its design was stable procurement of long-lead materials for the by its August 2009 production decision; however, the remaining 59 low-rate missiles. program has not completed developmental testing to prove that the design will meet performance and SM-6 capabilities will not be fully tested until after reliability requirements. Until developmental testing full-rate production is well underway. SM-6 is a key is completed, the risk of design changes remains. pillar of the Naval Integrated Fire Control-Counter Land-based developmental testing was successfully Air System—a system of systems designed to extend completed in 2009. However, sea-based the battle space over the horizon. A developmental developmental testing was suspended in May 2010 version of the integrated fire control capability was after two test failures. According to program demonstrated in 2009; however, it is not scheduled officials, the failures did not result in hardware to be demonstrated at-sea until fiscal year 2014. The design changes and developmental testing is program plans to have three of four full-rate scheduled to resume in January 2011. According to production lots under contract by that time. an official from DOD’s Office of the Director, According to the program office, the Office of the Operational Test and Evaluation, the program’s Secretary of Defense decided the SM-6 should be limited number of developmental test flights leaves fielded in advance of the Naval Integrated Fire little margin for error. Control-Counter Air System to address the Navy’s critical shortage of extended range missiles. Production Maturity The SM-6 is in low-rate production with 30 missiles Program Office Comments under contract and deliveries expected in March In commenting on a draft of this assessment, the  2011. We could not assess production maturity SM-6 program office disagreed with the GAO because the program did not provide statistical assertions that the program’s concurrent testing and process control data. However, prior to production production strategy puts the program at risk of cost start, the program demonstrated a maturity level growth and schedule delays; that the program has sufficient to proceed with low-rate production. To not completed testing to prove that the design meets obtain approval to begin full-rate production, the performance requirements; and that recent test program must achieve production maturity, failures increase the risk of design changes. successfully conclude at-sea developmental and According to the program office, the program operational flight testing, and demonstrate successfully completed reliability demonstration reliability. However, the program and DOD’s testing and has begun high-accelerated-life testing. Director of Operational Test and Evaluation have In addition, the SM-6 hardware is very mature, and not yet agreed to the flight reliability metrics and the risk of design changes is minimal. Officials failure definitions that will be used to assess the added that full testing of SM-6 using the legacy program. interface will be completed prior to the full-rate production decision and that flight testing using the Other Program Issues integrated fire control interface will take place prior The SM-6 program’s concurrent testing and to delivery of the first full-rate production missiles. production strategy puts the program at increased risk of cost growth and schedule delays if GAO Response unexpected design changes are required as a result Our reviews of DOD weapon systems confirm that of testing. In 2009, the program obtained approval fully configured, integrated, production- from the Under Secretary of Defense for Acquisition, representative prototypes should be tested before Technology and Logistics to begin low-rate initial committing to production. The benefits of testing production of up to 19 missiles before completing are maximized when the developmental tests are developmental testing. To minimize the risks completed prior to a production decision because inherent in this approach, the Under Secretary making design changes after production begins can required the program to complete developmental be both costly and inefficient.

Page 122 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: VTUAV Vertical Take-off and Landing Tactical Unmanned Aerial Vehicle (VTUAV) The Navy’s VTUAV will provide real-time imagery and data to support intelligence, surveillance, and reconnaissance requirements. A VTUAV system is composed of up to three air vehicles with associated sensors, two ground control stations, one recovery system, and spares and support equipment. The air vehicle launches and recovers vertically, and operates from ships and land. The VTUAV is being designed as a modular, reconfigurable system that supports various operations, including surface, antisubmarine, and mine warfare.

Source: © 2006 Northrop-Grumman Corporation.

Concept System development Production

Program Design Low-rate GAO Operational Initial Full-rate start review decision review test capability decision (1/00) (11/05) (5/07) (11/10) (9/11) (9/11) (2/12)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: Northrop Grumman As of Latest Percent Program office: Patuxent River, MD 12/2006 07/2010 change Funding needed to complete: Research and development cost $589.1 $664.8 12.8 R&D: $19.1 million Procurement cost $1,657.7 $1,881.3 13.5 Procurement: $1,660.2 million Total program cost $2,576.3 $2,547.2 -1.1 Total funding: $1,680.3 million Program unit cost $14.555 $14.555 0.0 Procurement quantity: 156 Total quantities 177 175 -1.1 Acquisition cycle time (months) 104 141 35.6

The VTUAV is in production, but the program may Attainment of Product Knowledge be at risk for further cost increases and schedule Production,288 delays as a result of concurrent testing and design, and technology production. The VTUAV program delayed the start maturity of operational test and evaluation by 2 years to e September 2011, due to system reliability and software maturity issues that were discovered Design and technolo192gy during developmental flight testing. During an maturity August 2010 flight test, the operator lost contact

with the aircraft, resulting in it entering restricted Desired level of knowledg air space. As a result, the program made changes to the software. In order to keep suppliers Technology producing at a minimum rate until operational test maturity 96 and evaluation is complete, the program increased its low-rate production quantities from 9 to 15. The program plans to achieve initial operational capability in September 2011 and full-rate production in February 2012—almost 2 years later 0 Development DOD Production GAO than previously planned. start design decision review (1/00) review (5/07) (11/10) (11/05) Page 123 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: VTUAV VTUAV Program from 9 to 15, in order to keep suppliers producing at a minimum rate until operational test and evaluation is complete. Technology Maturity The VTUAV relies on common, mature technologies. Other Program Issues The VTUAV program is currently considering a Design Maturity variety of future capabilities that could be added to The VTUAV design is still changing as a result of the system, including a surface search radar, a ongoing testing. The program had released all its signals intelligence package, an enhanced data and expected drawings by its May 2007 production communications relay, and weapons. The program decision, but it subsequently made design changes office had funding in place in fiscal year 2010 to and added drawings to address problems discovered integrate a surface search radar, but that funding during developmental testing. As of January 2011, was used to sustain the program when it 100 percent of the program’s total expected design experienced cost overruns resulting from software drawings have been released. However, the program issues discovered during developmental testing. is still making changes to the design as a result of Other planned capabilities are currently unfunded. concurrent testing and production. The program Work on these capabilities will be implemented as anticipates 20 changes in the aircraft’s design, subprograms. requiring 60 additional design drawings. The formal evaluation of the system’s operational suitability and Program Office Comments effectiveness was not completed as anticipated until In commenting on the draft of this assessment, the April 2010. During an assessment of the system on Navy stated that the VTUAV program has made the USS McInerney and subsequent developmental significant progress in the last year. VTUAV flight tests, the program discovered system conducted integration testing on the USS Freedom reliability and software maturity issues, including a in November 2010, is deployed on the USS weak communications link and false alarms related Halyburton, is being deployed to Afghanistan as part to the unmanned system’s status. More specifically, of the Intelligence, Surveillance and Reconnaissance during an August 2010 developmental flight test, the Task Force, and has been selected by Office of the operator lost contact with the aircraft, causing it to Secretary of Defense as the interim solution for a enter restricted air space. An investigation classified, maritime-based, intelligence, surveillance, determined the root cause to be a software design and reconnaissance urgent need requirement. flaw and the program has since made changes to the software. The VTUAV program also delayed the start VTUAV experienced a delay in the start of of operational test and evaluation by 2 years to operational evaluation and continues to make September 2011. The program now plans to achieve incremental strides to satisfy the full capability initial operational capability in September 2011 and production document requirements. Software full-rate production in February 2012—almost 2 changes to correct problems discovered during years later than previously planned. developmental testing have accounted for the majority of this schedule delay as any newly Production Maturity discovered software anomaly will delay flight testing The VTUAV was originally designed as a modified until a new software build is released. The commercial off-the-shelf item. We could not assess operational evaluation completion date is driven by production maturity because the program did not ship availability, and the USS Halyburton is not require the prime contractor or its supplier base to available until late summer 2011. The Navy also identify key product characteristics—the first step provided technical comments, which were to implementing production process controls. The incorporated as appropriate. program reports that one VTUAV supplier uses statistical process controls to measure elements of blade manufacturing. In addition, the program plans to develop production metrics based on critical safety items and safety of flight inspection criteria to support the full-rate production decision. The program increased its low-rate production quantities

Page 124 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: SSN 774 Virginia Class Submarine (SSN 774) The Navy’s Virginia class attack submarine is designed to combat enemy submarines and ships, fire cruise missiles, and provide improved surveillance and special operations support to enhance littoral warfare. The Navy awarded a Block III construction contract in 2008 and has begun construction on the first two hulls. In total, 7 ships have been delivered and 11 more are under contract. The Navy is introducing one new technology to improve system performance. We assessed this technology and made observations on design and production issues.

Source: U.S. Navy.

Concept System development Production

Development Development Full rate GAO Production 2 per year Block IV start– start— production review decision— production contract SSN 774 AESR decision AESR begins awarded (6/95) (12/08) (9/10) (11/10) (3/11) (2011) (2013)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: General Dynamics, As of Latest Percent Electric Boat Corporation 06/1995 09/2010 change Program office: Washington, DC Research and development cost $4,436.5 $7,076.9 59.5 Funding needed to complete: Procurement cost $55,113.7 $76,492.6 38.8 R&D: $709.7 million Total program cost $59,550.2 $83,569.4 40.3 Procurement: $40,375.8 million Program unit cost $1,985.005 $2,785.648 40.3 Total funding: $41,085.5 million Total quantities 30 30 0.0 Procurement quantity: 18 Acquisition cycle time (months) 134 151 12.7

The Virginia class submarine program was Attainment of Product Knowledge approved to enter full-rate production in Production,288 September 2010. Construction of the first two design, and technology Block III submarines has begun, and production of maturity the first hull featuring several affordability-based e design changes is underway. The Navy is also working to address quality control and reliability Design and technolo192gy concerns. The Navy will begin buying two maturity submarines per year in fiscal year 2011. It expects to realize its goal of reducing costs to $2.0 billion Desired level of knowledg (in fiscal year 2005 dollars) per ship by fiscal year 2012 ship procurement and hopes to further Technology decrease the time required to build each ship. The maturity 96 Navy has decided not to pursue two planned technology insertions for the Virginia class, but it is still developing advanced electromagnetic signature reduction (AESR) technology that will be introduced onto existing and new submarines. 0 Development DOD Production GAO AESR will begin testing in 2011. start design decision review (NA) review (NA) (11/10) (NA) Page 125 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: SSN 774

SSN 774 Program Other Program Issues The Navy is working to address quality control and reliability concerns. In November 2009, the Director, Technology Maturity Defense Research and Engineering (DDR&E), The Navy has decided not to pursue two planned highlighted several design and reliability-related technology insertions for the Virginia class, but it is deficiencies the program needed to address, but still developing advanced electromagnetic signature concluded they did not preclude the program from reduction (AESR) technology that will be introduced moving forward into full-rate production. These onto existing and new submarines. The Navy plans deficiencies, which included multiple subsystem to install AESR—software that monitors and failures, multiple “fail to sail” issues, and test aborts, optimizes the submarine’s signature—on ships were also cited by the Director, Operational Test and starting with SSN 782. The software will be installed Evaluation, as examples of the pervasive reliability on earlier ships over time. According to the Navy, problems that affect DOD systems. DDR&E also AESR prototype testing slipped by more than a year noted that the program did not have a reliability due to non-AESR-related schedule delays, and is measurement or growth program—a best practice. scheduled to begin on SSN 778 in September 2011. Navy officials told us that plans are in place to The Navy decided not to incorporate a conformal mitigate each of these issues. For example, acoustic velocity sensor wide aperture array on the according to the Navy, subsystem problems are ship after it found it would significantly increase, not being addressed with the vendors and shipyards decrease, life-cycle costs and complicate through changes to installation techniques, maintenance. The Navy is still evaluating more engineering changes or redesigns, and evaluations of affordable sail designs, but according to officials, the alternative technologies. Navy officials also told us larger, flexible payload sail is no longer being fail to sail events are not unexpected early in a considered because the communications program and that the Virginia class submarine has requirements that drove the need for more space not experienced any fail to sail events while have been eliminated. deployed. According to Navy and DDR&E officials, problems with a special hull treatment separating Prior to the program’s full-rate production decision, from the hull have also been mitigated by changing the Joint Requirements Oversight Council approved surface preparation techniques and redesigning a change to three Virginia class key performance coating molds. Delivered hulls will have the coating parameters. According to the Navy, they determined restored as needed, and more significant restoration that the original requirements were unrealistic and can occur during scheduled dry-dockings. According would not be worth the cost needed to achieve to Navy officials, this issue is not unique to the them. The change will not affect operations. Virginia class and has not resulted in any operational deficiencies. Navy officials said the shipbuilder has Design and Production Maturity also addressed the torpedo-room manufacturing The program was approved to enter full-rate quality issues that were identified in 2009. production in September 2010. The Navy will begin buying two ships per year in fiscal year 2011. The Program Office Comments Navy expects to achieve its goal of reducing costs to In commenting on a draft of this assessment, the $2.0 billion (in fiscal year 2005 dollars) per ship by program office provided technical comments, which fiscal year 2012 ship procurement and hopes to were incorporated as appropriate. reduce the time required to build each ship to about 60 months. Navy officials said construction of SSN 784—the first hull incorporating significant cost- reducing design changes—has been underway for approximately 2 years and is progressing well. The Navy expects the first few hulls with this new design to take longer to build, but expects to get back on schedule by the middle of Block III.

Page 126 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: WIN-T Increment 2 Warfighter Information Network-Tactical (WIN-T) Increment 2

WIN-T is the Army’s high-speed and high-capacity WIN-T Increment 2 – Initial Networking On The Move backbone communications network. WIN-T connects Army units with higher levels of command and provides the Army’s tactical portion of the

Global Information Grid. WIN-T was restructured WIN-T Inc 2 Tactical CommunicationCCommunicCoommunicationa s NodeNodeododede (While Mobile) (WhileWhileW Mobiloboobile)obileobibile)bibilebileilee) WIN-TWINWIW N-TTInT IncIn c 2 PointPoinoininntto ofof following a March 2007 Nunn-McCurdy unit cost PrePresencecee breach of the critical threshold, and will be fielded in Ruggedized four increments. The second increment will provide MPM-1000 Modem WIN-TWINWWIWIN-IN-TININ-TN-TTI IncIn nc2Tacc2Tacc2 2 TacticalTacctctiicicaicalcalccaaal Highband Networking Waveform CommunicationCCommunCommunicommmummunu ationt ns NodeNodeoodd the Army with an initial networking on-the-move Line of Sight Radio (While(Whileh Stationary) Stationartatiationarationaa y)y capability.

On-The-Move Highband SATCOM Networking Antenna WIN-TWINIINNTInNTINN-TTInT Inc Inc 2 2So Soldier Solddierieiere Waveform Line Initial Network NetworkN Extene sionioon of Sight Operations Antenna

Source: U.S. Army.

Concept System development Production

Program/development Design Low-rate GAO Full-rate Initial start review decision review decision capability (6/07) (2/08) (2/10) (11/10) (8/12) (5/13)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: General Dynamics As of Latest Percent C4 Systems Corp. 10/2007 12/2009 change Program office: Ft. Monmouth, NJ Research and development cost $235.0 $268.5 14.3 Funding needed to complete: Procurement cost $3,418.3 $4,469.9 30.8 R&D: $39.4 million Total program cost $3,653.3 $4,738.4 29.7 Procurement: $3,868.6 million Program unit cost $1.930 $2.138 10.8 Total funding: $3,908.0 million Total quantities 1,893 2,216 17.1 Procurement quantity: 1,856 Acquisition cycle time (months) 50 65 30.0

WIN-T Increment 2 entered production in Attainment of Product Knowledge February 2010 with all 15 of its critical Production,288 technologies mature. However, the program is design, and technology currently addressing significant performance and maturity reliability shortfalls that were revealed in a March e 2009 limited user test. The test results showed that WIN-T Increment 2 did not meet its operational Design and technolo192gy reliability requirements. DOD’s Director, maturity Operational Test and Evaluation, recommended

that the Army improve performance and training Desired level of knowledg to address these deficiencies and ensure success during initial operational test, which is scheduled Technology for early fiscal year 2012. We could not assess maturity 96 production maturity. According to the Army,  WIN-T is primarily an information technology integration effort that relies on commercially available products. Performance is measured Not assessed through a series of test events to demonstrate 0 Development DOD Production GAO performance at increasing levels of system start design decision review integration. (6/07) review (2/10) (11/10) (2/08) Page 127 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: WIN-T Increment 2 WIN-T Increment 2 Program three critical components demonstrated poor reliability. In a January 2010 operational assessment of this limited user test, DOD’s Director, Operational Technology Maturity Test and Evaluation (DOT&E), recommended that All 15 WIN-T Increment 2 critical technologies were the Army take certain actions to address these mature by its February 2010 production decision. In deficiencies to ensure success at the WIN-T September 2009, the Army completed a technology Increment 2 initial operational test scheduled for the readiness assessment to support a low-rate initial first quarter of fiscal year 2012. DOT&E production decision. An independent review team recommended that the Army improve the reviewed this technology readiness assessment and performance of the Increment 2 waveforms, provide the body of evidence used to support it and greater training to soldiers, refine its tactics and concluded that all 15 critical technologies were manning levels for Increment 2, and aggressively mature. In November 2009, the Director, Defense pursue a reliability growth program for WIN-T Research and Engineering, concurred with the Increment 2 components. According to the program independent review team’s assessment, noting that office, it is working to address these concerns and tests conducted by the Army show that each of  respond to these recommendations before the start WIN-T Increment 2’s critical technologies have been of the program’s operational test. demonstrated in a realistic environment. Program Office Comments Design Maturity In commenting on a draft of this assessment, the We could not assess the design maturity of WIN-T Army addressed (1) the development of a failure Increment 2 because the program office does not mode closure plan, (2) risk reduction events, and  track the number of releasable drawings. According (3) production qualification testing for WIN-T to the program office, WIN-T is primarily an Increment 2. With regard to the failure mode closure information technology integration effort that relies plan, the Army noted that, as directed by the Office on commercially available products. Performance is of the Secretary of Defense (OSD), the program measured through a series of component, office has identified, provided corrections for, subsystem, configuration item, and network level tested, and resolved all 37 of WIN-T Increment 2’s test events designed to demonstrate performance at identified failure modes identified from the limited increasing levels of system integration. Design user test. The Army also noted that a series of OSD- stability is measured through a problem tracking witnessed closure events and formal reports have report system. Problem tracking report trends are been completed. With regard to risk reduction, the reported and tracked on a weekly basis by the Army explained that, as directed by OSD, the program office. program office designed, executed, and performed analysis illustrating successful support for seven Production Maturity critical technical performance parameters, including We could not assess the production maturity of the mobile throughput and scalability. With regard to WIN-T Increment 2. According to the program office, production qualification testing, the Army noted that WIN-T is primarily an information technology the program office has two production qualification integration effort that relies on commercially test events scheduled in fiscal year 2012 to further available products. validate system readiness prior to the initial operational test. The Army also provided technical Other Program Issues comments, which were incorporated as appropriate. During a March 2009 limited user test, WIN-T Increment 2 failed to demonstrate the ability to support mobile operations as well as the required capabilities in forested terrain. WIN-T Increment 2 operational effectiveness was degraded because the program’s concept of operations, organizational structure, and manning were not adequate to operate and troubleshoot the network. Further, the test concluded that the WIN-T Increment 2 did not meet its operational reliability requirements because

Page 128 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: WIN-T Increment 3 Warfighter Information Network-Tactical (WIN-T) Increment 3 WIN-T is the Army’s high-speed and high-capacity WIN-T Increment 3 backbone communications network. WIN-T Full Networking On The Move Antenna connects Army units with higher levels of command 2 ch & 4 ch (High Band Radio Frequency Unit) JC4ISR and provides the Army’s tactical portion of the Air & Ground Ground Only Air Only Global Information Grid. WIN-T was restructured HRFU-MT HRFU-EK Addition of following a March 2007 Nunn-McCurdy unit cost Ground Only Airborne Tier breach of the critical threshold, and will be fielded in WIN-TWINWIN-WIIN-TINNN---TT Inc Innc 3 SoldierSSoldSoSololdldiiererr BCT DIV NetworkNNetwNetwoNetNeeetworketwotworkwoworkk Extentsionio ion four increments. The third increment will provide Single chassis JC4ISR Radio that provides 4x network capacity • Adds an airborne communications node as a third tier of tactical as the LOS Inc 2 networked communications. • Provides a fully mobile and flexible network to a dispersed force of the Army a full networking on-the-move capability. radio noncontiguous terrain. • Aerial tier increases throughput and reduces reliance on SATCOM. Full Network Operations

WIN-TWIN-WINWIINN-TN-TTTIn IncIn Inc 3 TacticalTTactccttiicicaicalcalcacal CommunicationComCCoCommuniCommunicCommCommunomommunicommuommunicmmunicmunimuni ationa ons NodeNodeddee (While(While(Whilehileilee Mobile)M obile)oobileobibile)bileile)ille)llee)e Fully automated network operations - no pause in operations

Source: U.S. Army.

Concept System development Production

Program/development GAO Design Low-rate Full-rate Initial start review review decision decision capability (7/03) (11/10) (4/13) (3/15) (7/18) (2/19)

Program Essentials Program Performance (fiscal year 2011 dollars in millions) Prime contractor: General Dynamics As of Latest Percent C4 Systems Corp. 05/2009 10/2010 change Program office: Ft. Monmouth, NJ Research and development cost $2,647.4 $2,189.2 -17.3 Funding needed to complete: Procurement cost $13,477.9 $11,476.8 -14.8 R&D: $1,046.9 million Total program cost $16,125.3 $13,666.0 -15.3 Procurement: $11,476.8 million Program unit cost $4.631 $4.261 -8.0 Total funding: $12,523.7 million Total quantities 3,482 3,207 -7.9 Procurement quantity: 3,168 Acquisition cycle time (months) 165 187 13.3

The WIN-T Increment 3 program will not fully Attainment of Product Knowledge mature its critical technologies until its planned Production,288 March 2015 production decision. Three of the design, and technology program’s 20 critical technologies are currently maturity mature and 15 are nearing maturity. Of the two e remaining technologies, one was rated as nearing maturity by an independent review team; but, the Design and technolo192gy Director, Defense Research and Engineering, maturity concluded that the technology’s ambiguous

requirements made it difficult to state whether it Desired level of knowledg had been adequately demonstrated. The other technology—a cryptographic device whose Technology development is being managed by the National maturity 96 Security Agency—has not been formally rated. The Army recently developed a revised WIN-T

Increment 3 acquisition program baseline to Projection account for changes due to the cancellation of the Not Not assessed assessed Future Combat System program. 0 Development GAO DOD Production start review design decision (7/03) (11/10) review (3/15) (4/13) Page 129 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: WIN-T Increment 3 WIN-T Increment 3 Program (NSA) manages the HAIPE program, and is responsible for certifying that vendors’ HAIPE products comply with the HAIPE interoperability Technology Maturity specification. According to an NSA representative, The WIN-T Increment 3 program will not fully NSA has informally assessed HAIPE 3.X and mature its critical technologies until its planned believes it is mature. However, NSA has not been March 2015 production decision. An April 2009 tasked by the Army or DDR&E with providing a review of the Army’s technology readiness formal assessment of HAIPE 3.X’s technology assessment for WIN-T Increment 3 by the Director, maturity. As a result, our assessment and Defense Research and Engineering (DDR&E), presentation of WIN-T’s Increment 3’s technology concluded that, of the program’s 20 critical maturity excludes this critical technology. technologies, 3 were mature and 15 were nearing maturity. Design Maturity We could not assess the design stability of the WIN-T Of the two remaining technologies, the Quality of Increment 3 because the program office does not Service Edge Device (QED) was rated as nearing track the number of releasable drawings. According maturity in the Army’s assessment; however, to the program office, this metric is not meaningful DDR&E concluded that this technology had because WIN-T is not a manufacturing effort, but ambiguous requirements that made it difficult to rather an integration effort. Performance is state whether it had been adequately demonstrated. measured through a series of component, DDR&E noted that while the Army had subsystem, configuration item, and network level demonstrated that the QED technology met test events designed to demonstrate performance at requirements under most conditions, in one increasing levels of system integration. Design stressing scenario, it did not. DDR&E stability is measured through a problem tracking representatives believe that it is unlikely that any report system. Problem track report trends are network can meet this requirement in all reported and tracked on a weekly basis by the environments. Since the QED technology was program office. shown to be robust and capable of meeting its requirement in most scenarios, DDR&E Other Program Issues recommended that the Army clarify the user’s In May 2009, the Under Secretary of Defense for requirements for this technology by the next design Acquisition, Technology and Logistics approved a review. According to a program office official, the revised acquisition program baseline for the WIN-T Army’s Training and Doctrine Command has Increment 3 program. However, this new baseline revisited user requirements for this critical was developed prior to the Secretary of Defense’s technology; however, a new rating for this recommended cancellation of the Future Combat technology will not be formalized until the program’s System (FCS) program, which was closely related to design review, currently scheduled for February WIN-T Increment 3. As a result, the Under Secretary 2014. restricted the Army from obligating or expending WIN-T Increment 3 funds associated directly with The other remaining technology—High Assurance FCS and directed that a new cost estimate and Internet Protocol Encryptor (HAIPE) version 3.X— acquisition program baseline be completed and was not available to be rated at the time of DDR&E’s approved. In October 2010, the Under Secretary review in April 2009. HAIPE is a device that encrypts approved a revised acquisition program baseline for and encapsulates Internet protocol packets so that WIN-T Increment 3, based on an independent cost they can be securely transported over a network of a estimate prepared by the Director, Cost Assessment different security classification. Version 1.3.5 of and Program Evaluation (CAPE), that reflects the HAIPE is mature; however, according to the WIN-T restructured program. program office, its use in WIN-T Increment 3 would result in a less efficient network design. DDR&E has Program Office Comments notified the Army that the maturity of the HAIPE version 3.X technology should be established to In commenting on a draft of this assessment, the DDR&E’s satisfaction before it is transitioned into Army provided technical comments, which were WIN-T Increment 3. The National Security Agency incorporated as appropriate.

Page 130 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: AMDR Air and Missile Defense Radar (AMDR) The Navy’s Air and Missile Defense Radar (AMDR) will be a next-generation radar system designed to provide ballistic missile defense, air defense, and surface warfare capabilities. AMDR will consist of an S-band radar for ballistic missile defense and air defense, X-band radar for horizon search, and a radar suite controller that controls and integrates the two radars. AMDR will initially support DDG 51 Flight III. The Navy expects AMDR to provide the foundation for a scalable radar architecture to defeat advanced threats.

Source: U.S. Navy. Current Status

The AMDR program entered technology development in September 2010 and is one of the first programs to incorporate affordability cost targets as part of the acquisition strategy. In addition, in September 2010, the Navy awarded three fixed-price incentive fee contracts to Northrop Grumman, Lockheed Martin, and Raytheon for S-band radar and radar suite controller technology development. The contractors will build and test prototypes to demonstrate the critical technologies during a 2-year technology development period. The Navy then plans to conduct a limited competition among the technology development contractors for engineering and manufacturing development. According to the program’s technology development strategy, the X-band radar technology is mature and the Navy plans to acquire it through full and open competition. The Navy will provide it as government-furnished equipment to the S-band and radar suite controller contractor to manage the integration during engineering and manufacturing development. Additional software development will be required to integrate the two radars.

To support the decision to enter technology development, the Navy conducted an early evaluation of technology maturity and identified six candidate critical technologies—four hardware-related and two software-related. According to program officials, digital beamforming—necessary for AMDR’s simultaneous air defense and ballistic missile defense mission—will likely take the longest time in development to mature. Program officials stated that while this technology is currently in use on existing radars, it has not been demonstrated on a large-aperture radar. The Navy is coordinating with the Air Force’s Space Fence program on the S-band radar’s technology development. The Navy estimates that AMDR will be available for delivery to a shipyard in fiscal year 2019.

Estimated Total Program Cost: $15,668.8 million  Research and development: $2,257.6 million  Procurement: $13,382.9 million  Quantities: 24

Next Major Program Event: Preliminary design review, July 2012

Program Office Comments: The AMDR program office concurred with this assessment and provided technical comments, which were incorporated as appropriate.

Page 131 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: B-2 DMS

B-2 Defensive Management System (DMS) Modernization

The Air Force’s B-2 DMS modernization is expected to upgrade the 1980s-era analog defensive management system to a digital capability. The modernization effort is intended to improve the frequency coverage and sensitivity of the electronic warfare suite, as well as update pilot displays and inflight replanning capability for avoiding unanticipated air defense threats. It is also expected to improve reliability and maintainability for the system. According to program officials, the current DMS is a major readiness driver for the aircraft.

Source: U.S. Air Force. Current Status

In June 2010, the B-2 DMS program received its material development decision and entered the material solution analysis phase. As part of this phase, the program will conduct an analysis of alternatives. According to program officials, the analysis of alternatives will consider three options: (1) a minimum upgrade (only critical electronic parts of current DMS); (2) a fully modernized DMS; and (3) an incremental DMS upgrade. Upon completion of the analysis of alternatives, the program anticipates entering the technology development phase in March 2011. The program tentatively plans to enter engineering and manufacturing development in 2013 after it completes its preliminary design review, and projects an initial operational capability for the B-2 DMS around fiscal year 2020.

A primary focus for the B-2 DMS program in technology development will be the low-observable antennas for the B-2’s leading edges. Overall, the program office has identified six critical technologies. According to the program office, five of the critical technologies are nearing maturity or are mature and have been demonstrated in a relevant or realistic environment. The low-observable antennas for the B-2 leading edges are the least mature. The primary risk with the technology’s development will be achieving the desired performance while still meeting the low-observable requirements of the aircraft. During the technology development phase, the program does not plan to develop prototypes of the full B-2 DMS, but instead plans to develop prototype antenna subsystems, which is the area it believes contains the most technological risk.

Estimated Total Program Cost: $1,505.8 million Research and development: $937.1 million Procurement: $568.7 million Quantities: 20 Note: This is an initial draft cost estimate developed by the program office. A formal baseline cost estimate is expected for the milestone A decision scheduled for March 2011.

Next Major Program Event: Technology development start (milestone A), March 2011

Program Office Comments: The program was provided a draft of this assessment and had no comments.

Page 132 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: B-2 EHF SATCOM Increment 2

B-2 Extremely High Frequency (EHF) SATCOM Capability, Increment 2

The Air Force’s B-2 EHF SATCOM is a satellite communication system designed to upgrade the aircraft’s ultra high frequency system to ensure continued secure, survivable communication. The system has three increments, with each expected to be its own program. Increment 2 is designed to provide connectivity by adding low-observable antennas and radomes to the aircraft. It also includes separate, nonintegrated Family of Advanced Beyond Line-of-Sight Terminals (FAB-T) and related hardware.

Source: U.S. Air Force. Current Status

The B-2 EHF Increment 2 program is in technology development. In March 2008, the program began a component advanced development effort that includes systems engineering, requirements analysis, technology maturation, and preliminary design activities. The program plans to enter engineering and manufacturing development in fiscal year 2013—3 years later than originally planned.

The B-2 EHF Increment 2 program has attempted to make decisions that balance requirements with technical solutions; however, antenna technology maturation and FAB-T availability still pose risks. For example, as a result of a recent trade study, the program’s key performance parameters were revised to reflect what is achievable and technically feasible. In addition, the Air Force changed the antenna location and technology to an active electronically scanned array (AESA) to lower risk. These decisions are expected to mitigate integration risks, but technology maturation risks still exist. The program expects its critical technologies to be nearing maturity by development start, but current technology readiness levels for AESA are relatively low and the program does not have a fallback antenna technology option should the AESA technology not mature as expected. An Air Force review also raised concerns that the decision to pursue AESA technologies exclusively may have precluded the use of lower risk, more affordable and mature technologies. In addition to these challenges, the availability of FAB-T hardware, which enables voice and data satellite communication, has significantly delayed the program. Due in part to the FAB-T delays, the B-2 EHF Increment 2 will not begin production by the current U.S. Strategic Command need date in fiscal year 2016. The program’s initial operational capability is expected in March 2020.

Estimated Total Program Cost: $1,796.7 million Research and development: $1,331.2 million Procurement: $464.5 million Quantities: 20

Next Major Program Event: Engineering and manufacturing development start, March 2013

Program Office Comments: In commenting on a draft of this assessment, the program office noted that a 2008 trade study validated program risk areas related to the antenna’s technology and planned location on the B-2. This study concluded that changing the antenna’s technology to an electrical antenna would considerably reduce risk, allowing relocation and, as a result, precluded negative effects on the B-2’s structure and radar cross section. The Air Force also provided technical comments, which were incorporated as appropriate.

Page 133 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: BMDS: ALTB BMDS: Airborne Laser Test Bed (ALTB) MDA’s ALTB, formerly the Airborne Laser program, is being developed as an advanced platform for the Department of Defense’s directed energy research program. The ALTB platform includes two solid state lasers and a high-energy laser housed aboard a modified Boeing 747-aircraft. The ALTB will transition to a national test platform over the next 4 years.

Source: Missile Defense Agency. Current Status

Until recently, MDA’s Airborne Laser program was developing a prototype system to negate enemy missiles during the boost phase of flight. However, after spending more than $5 billion on its development, DOD designated it as a test bed to demonstrate the potential of using directed energy as a viable technology against ballistic missiles. In February 2010, MDA demonstrated that the ALTB could successfully destroy a short- range threat-representative ballistic missile during the boost phase. According to the Director of Operational Test and Evaluation (DOT&E), the demonstration utilized a realistic target and was the most operationally relevant test of the Airborne Laser to-date. However, DOT&E also noted that despite the realism of the target missile, the test was not operationally realistic because a key component of the ALTB’s detection and tracking system was removed because it exhibited technical issues during earlier ground and flight testing. MDA conducted its second major flight test of the ALTB in early September 2010 against a short-range ballistic missile in the boost phase, but failed to destroy the target. The ATLB successfully detected and tracked the target, but corrupted beam control software steered the high-energy laser slightly off center. The ALTB safety system detected this shift and shut down the high-energy laser. MDA conducted a third flight test in October 2010 against a solid-fuel missile. However, while the system seems to have successfully acquired and tracked the plume or rocket exhaust of the target, it never transitioned to active tracking. As a result, the laser was not fired. The laser incorrectly reported that it was not ready and the safety default aborted the engagement.

Total Program Funding: NA

Next Major Program Event: NA

Program Office Comments: In commenting on a draft of this assessment, the ALTB program acknowledged DOT&E’s comments but noted that the test bed is focused on scientific learning rather than proof of operational capability. According to the program, MDA is collaborating with the Office of the Secretary of Defense to establish experiments that maximize benefits for DOD directed-energy programs. In fiscal year 2010, the ALTB conducted 40 flight experiments, including engagements of eight boosted targets. The ALTB also destroyed a solid-fuel metal-body target missile prior to its first major flight test. In May 2010, ALTB successfully engaged a diagnostic target missile at twice the range of the February experiments—a key risk- reduction event for future experiments. MDA continues to support directed energy as a hedge against the ever-changing threat environment.

Page 134 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: BMDS: FTF BMDS: Flexible Target Family (FTF) MDA’s Flexible Target Family was designed to be a family of short-, medium-, and long-range targets with common components for ground-, air-, and sea- launch capabilities. MDA is currently producing a groundlaunched, intermediate-range target, the LV-2, for BMDS flight tests. It is also developing a second target in this family, the 52-inch extended medium range ballistic missile target (eMRBM).

Source: Missile Defense Agency. Current Status

The LV-2 has been successfully tested and is in production. MDA flew the first LV-2 target vehicle in January 2010. During this test, the program was able to demonstrate all the critical technologies necessary for the  LV-2 in the current BMDS test plan. These technologies had not been flight tested in the necessary form, fit, and function before production began. There are currently five additional LV-2 target missiles in various stages of production, all in the same configuration. In order to assess production risks, the contractor is collecting data on labor hours and scrap, rework, and defect rates and has reported that these measures are tracking as expected. MDA had originally planned to produce at least one alternative configuration of the  LV-2 that used a shrouded reentry vehicle, but the mission requiring this technology was dropped from the agency’s test plan. MDA officials said they could purchase additional LV-2 targets, but no additional ones are currently under contract. The second LV-2 flight test was scheduled for the first quarter of fiscal year 2011.

MDA is not planning to conduct a risk-reduction flight for the second Flexible Target Family vehicle, the 52- inch eMRBM, because of budget constraints. This target uses more than 90 percent of the same ground and flight software, and 72 percent of the same components as the LV-2. However, these components must be incorporated into a new, smaller structure, which could pose an integration risk. MDA plans to use the first two eMRBMs in a BMDS operational test in 2012. In April 2010, MDA began acquiring the first three eMRBMs, and in September 2010 increased the quantity from three to five. MDA expects to complete negotiations and definitize the contract in July 2011.

Total Funding for LV-2 (Fiscal years 2005 to 2013): Research and development $622.2 million Quantities: 6

Next Major Program Event: Third LV-2 flight test, third quarter fiscal year 2011

Program Office Comments: In commenting on a draft of this assessment, MDA provided technical comments, which were incorporated as appropriate.

Page 135 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: C-27J JCA C-27J Joint Cargo Aircraft (JCA) The Air Force’s C-27J is a commercially available, multifunctional small-to-medium-size cargo aircraft. It is designed to help meet time-sensitive and mission critical transport requirements, and to augment the Air Force’s intratheater lift inventory. Its mission also includes casualty evacuation, airdrop, troop transport, aerial sustainment, and homeland security. The aircraft is capable of carrying up-armored High Mobility Multipurpose Wheeled Vehicles and heavy, dense loads such as aircraft engines and ammunition.

Source: L-3 Communications. Current Status

The C-27J Joint Cargo Aircraft program has been in production since June 2007. The sixth aircraft was delivered in November 2010. The program completed the transition from an Army-led joint program to an Air Force program in October 2010. Current plans are to procure 38 aircraft and assign them to Air National Guard bases. The Air Force Air Mobility Command and the National Guard Bureau have identified the first six locations to receive C-27J aircraft as Mansfield, Ohio; Baltimore, Maryland; Meridian, Mississippi; Battle Creek, Michigan; Fargo, North Dakota; and Bradley, Connecticut. The C-27J program office has established a foreign military sales section which has received requests for information about pricing and availability from a number of countries. However, as of July 2010, no foreign military sales cases had been initiated.

The C-27J program was to achieve initial operational capability in August 2010, but due to delays in multi- service operational test and evaluation, as well as delays in receiving certification from the Federal Aviation Administration, initial capability is now anticipated for January 2011. In June 2010 the Air Force received authorization from the Under Secretary of Defense for Acquisition, Technology and Logistics to expand the low-rate initial production procurement (LRIP) by up to 8 aircraft, to a total of 21. This does not affect the total quantities. Program officials explained that the firm fixed-price contract for the C-27J is based on prices established according to the number of aircraft purchased in a given period. By expanding the maximum number of LRIP aircraft, the program was able to order 8 aircraft in the latest ordering period at the currently agreed upon price rather than waiting for the next ordering period. According to program officials, this helped save an estimated $19 million and avoid a significant break in production.

Estimated Total Program Cost: $1,966.3 million Research and development: $118.3 million Procurement: $1,773.2 million Quantities: 38

Next Major Program Event: Full-rate production decision, February 2011

Program Office Comments: The program office was provided a draft of this assessment and did not offer any comments.

Page 136 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: DDG 51 DDG 51 Destroyer The Navy’s DDG 51 destroyer is a multimission surface ship designed to operate against air, surface, and subsurface threats. The Navy started buying DDG 51 destroyers in 1985 and expects to have 62 destroyers in service by 2012. In 2008, the Navy announced it would continue the program and plans to buy nine more ships over the next 5 years. The Navy then expects to start buying a new version of the ship—Flight III—in fiscal year 2016. Initial plans for Flight III include an increased emphasis on ballistic missile defense.

Source: U.S. Navy. Current Status

In 2008, the Navy announced its plan to restart the DDG 51 Flight IIA production line. The Navy anticipates that construction of DDG 113—the first ship in the restart program—will begin in fiscal year 2012 after an approximate 4-year gap in new DDG 51 starts. While program officials do not currently anticipate utilizing any new technologies or changing the Flight IIA design, the Navy could decide to use a hybrid electric drive designed to reduce fuel consumption on both new and existing DDG 51 destroyers, as well as in future designs, if it proves to be successful. The system is currently a prototype. The Navy is also working to address any industrial base issues that result from the production gap. For example, officials stated that before the Navy decided to restart DDG 51 production, the existing contractor for DDG 51 main reduction gears sold its production line. The Navy conducted a full and open competition and will now provide the gears as government-furnished equipment.

According to the program officials, a new air and missile defense radar will be the major technology effort for Flight III. The radar is being developed through a separate program office. According to the DDG 51 program, improving power generation on Flight III will be important to accommodate the expected increase in power and cooling requirements for this radar. The radar could also pose a risk for Flight III construction. The Navy estimates that it will not be available for delivery to a shipyard until fiscal year 2019—2 years prior to ship delivery according to the Navy. Officials stated that a decision has not been made to determine where the DDG 51 program will start in DOD’s milestone review process for the development and acquisition of the Flight III changes.

Estimated Total Program Cost (fiscal years 2010-2015): $16,926.5 million Research and development: $140.8 million Procurement: $16,785.7 million Quantities: 9

Next Major Program Event: DDG 113 construction start, fiscal year 2012

Program Office Comments: The program office provided technical comments, which were incorporated as appropriate.

Page 137 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: DWSS Defense Weather Satellite System (DWSS) DOD’s Defense Weather Satellite System (DWSS) is a next-generation polar-orbiting environmental satellite system with primary coverage in the early morning orbit. The system will incorporate data from the National Oceanic and Atmospheric Administration (NOAA) / National Aeronautics and Space Administration (NASA) Joint Polar Satellite System (JPSS) in the afternoon and from the European Meteorological Operational (MetOp) satellite in mid-morning.

Source: Northrop Grumman.

Current Status

In February 2010, the Executive Office of the President announced that the National Polar-orbiting Operational Environmental Satellite System (NPOESS) program was being restructured because of long- standing cost, schedule, and performance issues and management deficiencies. DOD and NOAA/NASA were directed to proceed with separately-managed acquisitions—DWSS for DOD and JPSS for NOAA/NASA. Currently, DOD obtains weather data through Defense Meteorological Satellite Program (DMSP) satellites. The Air Force launched a DMSP satellite in 2009 and has two remaining satellites, which it plans to launch as needed. To ensure continued coverage, the Air Force plans to have DWSS satellites available for launch in 2018 and 2021.

To reduce developmental risks and lower acquisition costs, the Air Force expects to leverage to the maximum extent the technology and investments made in the NPOESS program. The DWSS satellite bus is a modified NPOESS bus, but reduced in size and weight to meet DOD mission requirements. DWSS plans to use the visible infrared imager radiometer suite (VIIRS) sensor planned for NPOESS, as well as the ground control system based on the NPOESS design. The Air Force is evaluating microwave sensor alternatives, focusing on affordability while still providing DMSP-comparable microwave capability.

Estimated Total Program Cost: TBD Research and development: TBD Procurement: NA Quantities: 2

Next Major Program Event: Materiel development decision, February 2011

Program Office Comments: Program officials stated they are working on restructuring the NPOESS prime contract to facilitate transition of civil NOAA/NASA work while focusing on DOD mission needs. Also, DOD and NOAA/NASA are working to complete a memorandum of agreement on sharing common elements at the VIIRS contractor and DOD plans to acquire duplicate test sets and ground support equipment to minimize single supplier issues.

Page 138 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: EPS Enhanced Polar System (EPS) The Air Force’s Enhanced Polar System (EPS) will provide next-generation protected extremely high frequency (EHF) satellite communications in the polar region above 65 degrees northern latitude. EPS will replace the current Interim Polar System and serve as a polar adjunct to the Advanced EHF (AEHF) system. EPS consists of two EHF payloads hosted on classified satellites. A gateway will connect modified AEHF communications terminals to other communication systems utilizing an extension of the AEHF mission control segment.

Source: LinQuest Corporation. Current Status

The EPS program is preparing to enter system development. It was initiated in fiscal year 2006 to fill the gap left by the cancellation of the Advanced Polar System. In December 2007, the Under Secretary of Defense for Acquisition, Technology and Logistics directed the program to bypass concept development and proceed to a system development decision in order to synchronize the program’s schedule with the host satellite production timeline. Since then, the Air Force has determined that the EPS program would require additional payload engineering and changes to the gateway, and the program’s entry into system development has been delayed from February 2010 to the third quarter of fiscal year 2011. In addition, the program’s projected cost could significantly increase. According to program officials, the EPS program did not have an opportunity to develop a complete and thorough cost estimate because it was directed to proceed directly to a system development decision. The fiscal year 2011 President’s budget request included about $1 billion for the program through fiscal year 2015; however, program officials stated that the new cost estimate might require double this amount. Before the EPS program can enter system development, the milestone decision authority will have to certify that this funding is available.

According to the program office, EPS critical technologies are mature. Although the program does not plan to conduct prototyping prior to the start of development, program officials noted the EPS has conducted prototyping in the payload, ground, gateway, and terminal segments as part of the technology development phase of the program. According to DOD acquisition policy, the technology development strategy for each major defense acquisition program shall provide for prototypes of the system or, if a system prototype is not feasible, for prototypes of critical subsystems before it gets approval to enter the engineering and manufacturing development phase.

Estimated Total Program Cost: $1,490.7 million Research and development: $1,395.4 million Procurement: $79.7 million Quantities: 2

Next Major Program Event: System development decision, third quarter of fiscal year 2011

Program Office Comments: The EPS program office provided technical comments, which were incorporated as appropriate.

Page 139 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: F-22A Raptor F-22A Raptor The Air Force’s F-22A Raptor is the only fifth- generation operational air-to-air and air-to-ground fighter/attack aircraft. This aircraft integrates stealth, supercruise, and advanced avionics, maneuverability, and weapons in one platform. The Air Force established the F-22A modernization and improvement program in 2003 to add enhanced air- to-ground, information warfare, reconnaissance, and other capabilities and improve the reliability and maintainability of the aircraft.

Source: U.S. Air Force. Current Status

In April 2009, the Secretary of Defense announced that F-22A production would end at 187 aircraft, and as of September 2010, the Air Force had accepted delivery of 160 aircraft. According to the Director, Operational Test and Evaluation, the F-22A could have difficulty meeting operational suitability requirements relating to low-observable maintainability when the system reaches full maturity. Air Force officials reported that the low-observable materials are difficult to manage and maintain, requiring nearly twice the number of maintenance personnel as anticipated.

The Air Force is upgrading the F-22A fleet in four increments. The first increment has been fielded. The second increment—Increment 3.1—will begin follow-on operational test and evaluation in January 2011. These increments add enhanced air-to-air and air-to-ground; intelligence, surveillance, and reconnaissance; and synthetic aperture radar capabilities. The third increment—Increment 3.2—has been divided into two phases. The first will deliver electronic protection, combat identification, and Link 16 communication upgrades beginning in 2014. The second phase will begin fielding AIM-9X and AIM-120D capabilities, and additional electronic protection upgrades beginning in 2016. The multifunction advanced datalink, planned to provide interoperability with the F-35 Joint Strike Fighter, was removed from Increment 3.2 after costs increased significantly over initial estimates and the program was unable to secure fiscal year 2010 funding. Increment 3.3 will enable compliance with air traffic management standards, but the additional content for this increment has not yet been determined.

Estimated Total Program Cost: $77,392.8 million Research and development: $39,171.7 million Procurement: $37,560.5 million Quantities: 188

Next Major Program Event: Final aircraft delivery, February 2012

Program Office Comments: The Air Force acknowledged challenges associated with maintaining a fifth- generation low-observable system, and said there are currently more than 10 ongoing efforts under the reliability and maintainability maturation program to increase maintainability and material durability. The Air Force also stated that the majority of repairs to the low observable system resulted from having to perform other maintenance activities, rather than to address problems with the low observable system itself. According to the service, all operational units have reported mission capable rates for the low-observable system of about 90 percent, since January 2009. The Air Force also provided technical comments, which were incorporated as appropriate.

Page 140 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: H-1 Upgrades H-1 Upgrades (UH-1Y/AH-1Z) The Navy’s H-1 Upgrades Program converts the  AH-1W attack helicopter and the UH-1N utility helicopter to the AH1-Z and UH-1Y configurations, respectively. The mission of the AH-1Z attack helicopter is to provide rotary-wing fire support and reconnaissance capabilities in day/night and adverse weather conditions. The mission of the UH-1Y utility helicopter is to provide command, control, and assault support under the same conditions.

Source: U.S. Navy. Current Status

The UH-1Y and AH-1Z configurations are in production. DOD approved full-rate production for the UH-1Y in September 2008. The program received approval to enter full-rate production for the AH-1Z in November 2010. The Navy completed operational testing and evaluation for the AH-1Z in June 2010. The evaluation report, issued in September 2010, concluded that the AH-1Z is operationally effective and suitable, and recommended the aircraft for introduction in the fleet. The report also highlighted concerns with logistics supportability and recommended this deficiency be corrected prior to the first AH-1Z operational deployment.

In July 2010, the Deputy Commandant of the Marine Corps for Aviation issued a directive changing the composition of H-1 program procurements from 123 UH-1Ys and 226 AH-1Zs to 160 UH-1Ys and 189 AH-1Zs. The Deputy Commandant cited increased demand for utility helicopters in support of combat operations and the increase in combat power of the AH-1Z compared to the AH-1W as support for the directive. As of November 2010, there were 98 H-1 upgrades under contract, and 46 aircraft—34 UH-1Ys and 12 AH-1Zs—had been delivered to the fleet.

Estimated Total Program Cost: $11,866.5 million Research and development: $1,855.3 million Procurement: $10,011.2 million Quantities: 4 (Research and Development), 349 (Procurement)

Next Major Program Event: Initial operational capability, AH-1Z, second quarter fiscal year 2011

Program Office Comments: The Navy provided technical comments, which were incorporated as appropriate.

Page 141 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: JLTV Joint Light Tactical Vehicle (JLTV) The Army and Marine Corps’ Joint Light Tactical Vehicle is a family of vehicles focused on balancing personnel protection, payload, and performance. JLTV is expected to reduce life-cycle costs through commonality at the subassembly and component level. The JLTV is expected to provide defensive measures for troops while in transport, increase payload capability, improve the logistics footprint, and reduce soldier workload associated with system operation and field maintenance activities.

Source: Department of Defense. Current Status

In December 2007, the Under Secretary of Defense for Acquisition, Technology and Logistics directed the Army to begin a 27-month technology development phase for the JLTV program with the goal of reducing risks prior to and shortening the length of system development. Prior to entering engineering and manufacturing development, the JLTV program will develop prototypes, demonstrate critical technologies in a relevant environment, and conduct a preliminary design review, as required by DOD policy. In October 2008, the Army awarded three technology development contracts. The three contractors delivered prototype vehicles in May 2010. Testing on the prototypes has begun and is expected to last about a year. At the conclusion of technology development, the Army plans to hold a full and open competition and award two engineering and manufacturing development contracts. One of these two contractors will be selected for production.

The JLTV’s acquisition costs have not been determined, but based on evolving user requirements the JLTV average unit manufacturing cost is estimated at $356,000. This cost does not include contractor general and administrative cost, contractor profit, or government-furnished equipment. The program has a demanding set of projected requirements and tradeoffs may be necessary. Among the program’s key challenges is whether the vehicle can provide the performance and reliability required within the weight limits for helicopter transport.

Estimated Total Program Cost: $53,523.3 million Research and development: $1,082.2 million Procurement: $52,298.3 Quantities: 60,383

Next Major Program Event: Engineering and manufacturing development start, January 2012

Program Office Comments: In commenting on a draft of this assessment, the program office stated that JLTV provides a design that supports mobility, reliability, and maintainability within weight limits to ensure transportability to and from the battlefield. The program office also noted that JLTV uses scalable armor solutions to meet requirements for added protection while maintaining load carrying capacity.

Page 142 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: KC-X KC-X Program The Air Force’s KC-X program is the first of three phases in the recapitalization of the KC-135 aerial refueling tanker fleet. The KC-X acquisition strategy calls for the procurement of 179 commercial derivative aircraft tankers at an expected cost of around $35 billion. The KC-X is planned to provide sustained aerial refueling capability to facilitate global attack, air-bridge, deployment, sustainment, employment, redeployment, homeland defense, theater support, and specialized national defense missions. Source: U.S. Air Force. Note: Photo is of the KC-135 Stratotanker, the aircraft the KC-X will replace. Current Status

The KC-135 recapitalization effort is the Air Force’s highest acquisition priority. It is expected to involve the procurement of about 600 aircraft over a 40-year period with a cost that could exceed $100 billion. The first phase of this recapitalization—the KC-X program—is in source selection. According to the Office of the Secretary of Defense, the Air Force plans to award a fixed-price incentive fee contract for the engineering and manufacturing development phase of the KC-X program in early fiscal year 2011. The request for proposal for this contract requires all critical technologies to be at a technology readiness level 6— demonstrated in a relevant environment—or higher. According to its current acquisition strategy, the primary technical risk for the KC-X program is the integration of military hardware and software on a commercial platform. The Air Force plans to begin procurement of four aircraft for development and testing purposes in fiscal year 2011.

The Air Force has experienced numerous delays in awarding a development contract. In February 2008, the Air Force awarded a development contract to Northrop Grumman and the European Aeronautic Defense and Space Company. However, the contract award was the subject of a bid protest that was sustained by GAO. As a result, in October 2008, the Office of the Secretary of Defense directed the Air Force to terminate the contract and conduct a new competition.

Estimated Total Program Cost: TBD Research and development: TBD Procurement: TBD Quantities: 179

Next Major Program Event: Engineering and manufacturing development start, early fiscal year 2011

Program Office Comments: The Office of the Director of Defense Procurement and Acquisition Policy provided technical comments on a draft of this assessment, which were incorporated as appropriate.

Page 143 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: NPOESS National Polar-orbiting Operational Environmental Satellite System (NPOESS) Until it was significantly restructured in February 2010, NPOESS was a triagency program with the Department of Commerce (National Oceanic and Atmospheric Administration), DOD (Air Force), and the National Aeronautics and Space Administration (NASA) designed to merge civil and defense weather satellite programs to reduce costs, provide global weather and climate coverage, and improve capabilities above current fielded systems.

Source: Northrop Grumman. Current Status

In February 2010, the Executive Office of the President announced that the NPOESS program was being restructured because of long-standing cost, schedule, and performance issues and management deficiencies. As part of the restructuring, DOD, NOAA, and NASA were directed to proceed with separately-managed acquisitions—the Defense Weather Satellite System (DWSS) and the civilian Joint Polar Satellite System (JPSS). DOD, NOAA, and NASA will still be responsible for meeting NPOESS weather and climate observational requirements in their assigned orbits. DOD will be responsible for collecting weather and climate data during the early morning orbit. NOAA and NASA will be responsible for collecting weather and climate data during the afternoon orbit. These agencies are expected to share data from a common ground system to be managed by NOAA and NASA.

According to the Executive Office of the President, the JPSS will consist of platforms based on the NPOESS Preparatory Project (NPP) satellite. NOAA is currently developing its plan for the JPSS and is considering options such as developing a smaller satellite than the one planned for NPOESS and removing sensors that were planned for the NPOESS satellites. NOAA will have to manage a series of challenges during the transition from NPOESS to JPSS, including the resolution of existing contracts and intellectual property issues and the loss of skilled workers. NOAA also has to address risks related to the readiness of ground systems to support the planned October 2011 NPP satellite launch and the instrument readiness to support a JPSS launch in 2014.

DOD is in the early stages of planning its approach for the DWSS program. See page 138 for more information about the status of the program.

Estimated Total Program Cost: According to program officials, the estimated total program cost for NPOESS was about $5.3 billion when last reported in fiscal year 2010.

Next Major Program Event: NA

Program Office Comments: We did not request comments on this assessment because the NPOESS program office was disbanded before we completed our review.

Page 144 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Navy UCAS-D Navy Unmanned Combat Air System Aircraft Carrier Demonstration (UCAS-D) The Navy’s UCAS-D program plans to demonstrate and mature technologies that could be used on a future unmanned, long-range, low-observable carrier-based aircraft with an autonomous air- refueling capability, which would provide greater standoff capability, expanded payload and launch options, and increased naval reach and persistence. Following completion of the demonstration and technology maturation effort, the Navy will decide whether to initiate a formal acquisition program.

Source: Northrop Grumman. Current Status

The Navy UCAS-D program is a demonstration program. The program plans to demonstrate carrier operations, including autonomous aerial refueling, of a low-observable unmanned combat air system and mature both aircraft and other critical technologies needed to operate and integrate the aircraft with the ship. These efforts are intended to support an acquisition decision for a potential Navy UCAS major defense acquisition program, which could enter the acquisition process at either the technology demonstration or engineering and manufacturing development phase. In August 2007, the Navy awarded a cost-plus incentive fee contract to Northrop Grumman for the design, development, integration, test, and demonstration of two unmanned combat air systems. The contractor has completed the first air vehicle and is currently building the second air vehicle. First flight is scheduled for December 2010, and according to program officials, the demonstration effort—excluding the autonomous aerial refueling capability—is scheduled to be completed by 2013. The refueling capability is not scheduled to be demonstrated until summer 2014 and all technology maturation and demonstration efforts are to be completed by 2015.

While the prime contractor’s estimated costs have grown by over $200 million to $813 million, according to the program manager, there is sufficient funding in the President’s fiscal year 2011 budget request to complete the demonstration program. The cost of the prime contract has grown, in part, because the contractor originally proposed completing the demonstration sooner than 2013 and the Office of the Secretary of Defense added an autonomous aerial refueling demonstration to the program in 2008. Although the Navy officials initially agreed to the accelerated schedule, it was subsequently determined that the technology involved was more complex than originally anticipated. After a 2010 review by the Navy, the program’s cost and schedule estimates were revised to reflect the Navy’s original 2013 program completion date as well as the addition of the autonomous aerial refueling capability.

Estimated Total Program Cost: $1,562.1 million Quantities: 2

Next Major Program Event: First flight, December 2010

Program Office Comments: The Navy UCAS-D program office provided technical comments, which were incorporated as appropriate.

Page 145 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Nett Warrior Nett Warrior Increment I The Army’s Nett Warrior program is an integrated ground soldier system designed to provide embedded training and increased situational awareness, lethality, mobility, survivability, and sustainability during combat operations. There are three increments planned. Increment I will focus on developing the situational awareness system used initially with Stryker Brigade Combat Teams. The program is a descendant of the Land Warrior program, which was terminated due to cost concerns and funding constraints.

Source: U.S. Army. Current Status

The Nett Warrior program was approved to enter technology development in February 2009 and plans to proceed directly to production in fiscal year 2011. The Army has awarded cost-plus-fixed-fee contracts to General Dynamics, Raytheon, and Rockwell Collins for prototypes, which include a hands-free display, headset, computer, navigation equipment, antennas, and cables. These prototype designs have undergone developmental testing and are currently being evaluated in a limited user test. The Army has identified five critical technologies for Nett Warrior Increment I—energy/power management subsystem, antenna, navigation, user controller, and voice intelligibility. These technologies are currently nearing maturity and, pursuant to a program acquisition decision memorandum, they must be demonstrated in an operational environment prior to exiting technology development. Based on the results of the limited user test, conducted at Ft. Riley, Kansas, and a technology readiness assessment of the program by the Director, Defense Research and Engineering, the Under Secretary of Defense for Acquisition, Technology and Logistics will make a decision on whether the program will proceed to engineering and manufacturing development or directly to production and deployment as planned. While it is not designated as a critical technology, the fully networked capability of the Nett Warrior Increment I will not be achieved until the Joint Tactical Radio System is incorporated after full-rate production.

Estimated Total Program Cost: $1,669.4 million Research and development: $179.8 million Procurement: $1,489.6 million Quantities: 20,430

Next Major Program Event: Low-rate initial production decision, fiscal year 2011

Program Office Comments: The program office provided technical comments, which were incorporated as appropriate.

Page 146 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: OR Ohio-Class Replacement (OR) / Sea Based Strategic Deterrent The Navy’s Ohio-class Replacement (OR) / Sea Based Strategic Deterrent will replace the current fleet of Ohio-class ballistic missile submarines (SSBN) as they begin to retire in 2027. The Navy began research and development in 2008, in order to avoid a gap in sea-based nuclear deterrence between the Ohio class’s retirement and the production of a replacement. The Navy is working with the United Kingdom to develop a Common Missile Compartment (CMC) for use on the Ohio-class replacement and the United Kingdom’s replacement for the Vanguard SSBN.

Source: General Dynamics Electric Boat. Current Status

The OR program was scheduled to enter technology development in December 2010. The Navy’s fiscal year 2011 long range shipbuilding plan includes 12 OR SSBNs and projects that the program will receive authorization to begin construction on the lead ship in fiscal year 2019. The high expected cost of the OR has been an early focus of the program. Both DOD and Navy officials have stated the cost of the program could dominate Navy shipbuilding budgets in the 2020 to 2030 time frame. The size and number of missile tubes is one of a number of cost drivers. The OR’s analysis of alternatives, which was approved by the Office of the Secretary of Defense’s office of Cost Analysis and Program Evaluation in December 2009, discussed the size and number of missile tubes; however, a final decision is still pending. According to the program office, Northrop Grumman, Babcock Integrated Technologies, Electric Boat Corporation, and BAE Systems Marine have been awarded contracts for assembly of prototype missile tubes.

The main focus of OR research and development to date has been the CMC. The United Kingdom has provided $329 million for this effort since fiscal year 2008. During fiscal years 2009 and 2010, the Navy had allocated about $183 million for the design and prototyping of the missile compartment. According to Navy officials, the program’s other areas of emphasis include developing stealth technologies, ensuring a balanced ship design and the production readiness of the missile tube manufacturing base, and strengthening government system engineering personnel.

Overall, the Navy has received $510.3 million for the OR program over fiscal years 2009 and 2010. The President’s budget request for fiscal year 2011 included an additional $672.3 million. These numbers include funding for OR and propulsion plant development.

Estimated Total Program Cost: TBD

Next Major Program Event: Engineering and manufacturing development start, fiscal year 2015

Program Office Comments: The OR program generally concurred with this assessment. The program stated that over the past year, it has focused on containing nonrecurring engineering and construction costs by incorporating innovations to ship design and construction methodologies and practices as well as lessons learned from the Virginia-class program’s design and construction. The program office also provided technical comments, which were incorporated as appropriate.

Page 147 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: Space Fence Space Fence The Air Force’s Space Fence will be a new system of ground-based radars to replace the aging Air Force Space Surveillance System. It will use higher radio frequencies to detect and track smaller Earth- orbiting objects. The system will consist of up to two geographically dispersed radars (notionally located in Australia, Ascension Island, or Kwajalein Atoll) to help ensure effective space surveillance coverage. The system’s enhanced capabilities are expected to significantly increase the number of orbiting objects detected and tracked.

Source: U.S. Air Force. Current Status

The Space Fence program began technology development in March 2009. In June 2009, the Air Force awarded three $30 million firm fixed-price contracts to Lockheed Martin, Northrop Grumman, and Raytheon. The Northrop Grumman contract was subsequently terminated after a reduction in program funding. The program is currently focused on making cost, schedule, and performance tradeoffs to ensure its affordability. The Air Force next plans to conduct a full and open competition and award up to two contracts for a maximum of $214 million to continue technology development. The contracts will go through the preliminary design review, which is planned for January 2012, followed by another full and open competition leading to a single, final development and production contract in July 2012. The first Space Fence radar site is scheduled to provide initial operational capability by the end of fiscal year 2015, with the final site providing full capability by 2020.

The program office identified five critical technologies and expects them to be nearing maturity by the preliminary design review. Mature backup technologies exist; however, all have potentially higher acquisition or operating costs associated with them, or both. According to the program office, using these backups could make the program unaffordable.

According to the program office, a separate program for the development of a new space command and control system poses a risk for Space Fence. This new system, which will process Space Fence data, needs to be available when the Space Fence is fielded because the amount of data it will generate exceeds existing command and control system performance limits.

Estimated Total Program Cost: $2,717.6 million Research and development: $1,682.2 million Procurement: $1,035.4 million Quantities: 1 (Research and Development), 1 (Procurement)

Next Major Program Event: Final development and production start, July 2012

Program Office Comments: The program office stated that changes to the acquisition strategy increase competition, induce further contractor design and cost realism, and reduce program risk to deliver the first system capability by September 2015. The preliminary design review is expected to occur 6 months prior to the planned June 2012 milestone B, allowing for a more informed decision at final contract award. The program office also provided technical comments, which were incorporated as appropriate.

Page 148 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: SMOD Stryker Modernization (SMOD) The Army’s Stryker family of vehicles is a group of deployable, wheeled, armored vehicles. The Stryker upgrade program, commonly known as Stryker Modernization, is a technological overhaul of targeted components designed to improve the vehicle’s survivability, mobility, lethality, and networking capabilities. The system enhancements, which will be applied to all 10 Stryker variants, will enable the vehicles to remain in operation through 2050. The program will comprise two increments. We made observations on increment one.

Source: U.S. Army. Current Status

The Army is planning a materiel development decision (MDD) for the Stryker Modernization program, which will determine where it will enter the acquisition process. The program is the outgrowth of a modernization concept that was briefed at a Stryker configuration steering board meeting in August 2009. According to the Stryker program, the modernization program is necessary to address space, weight, and power constraints that could affect the vehicles’ ability to meet current and future warfighting needs. Before moving forward with the program, the Army had to await the outcome of two events. In March 2010, the Under Secretary of Defense for Acquisition, Technology and Logistics directed the Army to provide a complete overview of Stryker Modernization efforts, including an evaluation of how Stryker Modernization activities fit into the Army strategy for meeting future ground combat vehicle requirements. In addition, the Army must complete its combat vehicle portfolio review.

The first increment of the Stryker Modernization program will consist of two phases. According to program officials, phase 1 will focus on the common characteristics of all 10 variants, such as the chassis and drive train, spare parts, and test equipment. Phase 2 will focus on enhancing characteristics unique to each of the variants, such as fire support and medical evacuation. The Army has identified three critical technologies for phase 1—semiactive suspension, survivability improvements, and power generation. According to the program, all three technologies will be demonstrated in a relevant or realistic environment before the program enters engineering and manufacturing development. The program also plans to select, through competition, three contractors to develop prototypes of key subsystems prior to this phase. Additional critical technologies may be identified for phase 2. The Stryker Modernization strategy will be determined by Army senior leaders and then that strategy will be recommended to the defense acquisition executive at the program’s MDD for his approval.

Estimated Total Program Cost: $2,179.8 million Research and development: $1,190.0 million Procurement: $989.8 million Quantities: 35 (Research and Development), 664 (Procurement)

Next Major Program Event: Materiel development decision, TBD

Program Office Comments: In commenting on a draft of this assessment, the Army provided technical comments, which were incorporated as appropriate.

Page 149 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: 3DELRR Three Dimensional Expeditionary Long Range Radar (3DELRR) The Air Force’s 3DELRR will be the long-range, ground-based sensor for detecting, identifying, tracking, and reporting aircraft and missiles for the Joint Forces Air Component Commander. It will provide real-time data and support a range of expeditionary operations in all types of weather and terrain. It is being acquired to replace the Air Force’s AN/TPS-75 radar systems. The Marine Corps is considering 3DELRR as a potential replacement to the AN/TPS-59 to support the Marine Air-Ground Task Force Commander.

Source: U.S. Air Force. Current Status

The 3DELRR program was approved to enter the technology development phase in May 2009. The Air Force has awarded firm fixed-price contracts for the development of system-level prototypes. The program expects to demonstrate the capability of each of the prototypes by the end of the first quarter of fiscal year 2011. The program plans to conduct a full and open competition and award a single cost plus incentive fee contract for program definition and risk reduction before seeking approval to enter engineering and manufacturing development in the third quarter of fiscal year 2013. Initial operational capability for the radar is targeted for approximately 2019.

The 3DELRR program is focused on reducing technical risk before beginning system development. The program has identified six critical technologies. The program’s technology development strategy calls for these technologies to be nearing maturity and demonstrated in a relevant environment by the start of system development. The program also expects to complete a preliminary design review prior to development start and a critical design review early in the engineering and manufacturing development phase.

According to the program office, the primary risks going forward are securing adequate funding and accurately capturing the user’s requirements. Additional near-term funding is necessary to execute the current acquisition strategy. In the event of a program budget shortfall, the program office would have to examine alternatives including revising the acquisition strategy, conducting requirements tradeoffs, and extending the delivery schedule. Further, if the program’s requirements document does not accurately represent user requirements, then the system design could require changes late in the acquisition process.

Estimated Total Program Cost: $2,092.4 million Research and development: $743.3 million Procurement: $1,349.1 million (Air Force only) Quantities: 35 (Air Force only)

Next Major Program Event: Program definition and risk reduction contract award, second quarter fiscal year 2012

Program Office Comments: The program office concurred with the assessment and provided technical comments, which we incorporated as appropriate.

Page 150 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: V-22 Osprey V-22 Joint Services Advanced Vertical Lift Aircraft (Osprey) The V-22 Osprey is a tilt-rotor aircraft developed for use by the Marine Corps, Air Force, and Navy. The Marine Corps and Air Force Special Operations Command have completed multiple deployments of the MV-22 and CV-22, respectively, in Iraq, Africa, Afghanistan, South America, and in support of the humanitarian efforts in Haiti. As of December 2010, there were 125 V-22s in service. The program is currently focused on improving readiness, decreasing ownership cost, and preparing for a second multi-year procurement contract.

Source: U.S. Marine Corps. Current Status

The V-22 has been deployed and is supporting combat operations; however, the aircraft has experienced reliability and readiness issues that have resulted in high maintenance and operating costs. Since 2009, the program has undertaken several initiatives to understand and reduce operations and maintenance costs after it became apparent the MV-22 was exceeding its budgeted cost per flight hour. The program developed a new cost model to increase the quality and accuracy of cost per flight hour estimates and established a team to identify cost savings opportunities. According to the Marine Corps, it is also working with industry to increase the durability of key components that affect readiness and operating costs. One of the program’s top priorities is the ice protection system. A June 2010 Air Force accident report on a March 2009 CV-22 engine failure concluded that it was caused by a part from the central deice distributor (an ice protection system component) shaking loose and falling into the engine. The report stated that this part had a history of structural issues related in part to its design. The program office has addressed the risks identified in the report by instituting an inspection of this component at the 35 hour interval and installing a more durable version of the part on aircraft in production. According to the program office, all CV-22s and deployed MV-22s have been retrofitted with the hardware fix and retrofit kits are available for the remaining fleet. In April 2010, a CV-22 crashed on an infiltration mission in support of ground forces in Afghanistan. In August 2010, the Air Force Accident Investigation Board reported that there were 10 substantially contributing factors to the accident and that they fell into four categories: mission execution, environmental conditions, human factors, and aircraft performance. We were unable to follow-up on how these factors are being addressed during our assessment. The V-22 program is in the fourth year of its first multiyear procurement contract. The program will request authority to enter, and is planning and budgeting for, a second multiyear procurement contract to begin in fiscal year 2013.

Estimated Total Program Cost: $56,061.1 million Research and development: $13,114.7 million Procurement: $42,829.3 million Quantities: 458 aircraft

Program Office Comments: In its comments on a draft of this assessment, the program office stated that the V-22 has met all operational tasking and the Marines and Air Force are exceptionally satisfied with the  V-22’s “game changing” capabilities. It also emphasized the aircraft’s exceptional survivability and the numerous improvements that have been made to it in the last decade. Eight Marine and two Air Force Osprey squadrons are operational. The program projects reaching 100,000 flight hours in early 2011. Production is fully mature with aircraft deliveries on or ahead of schedule. The program’s overall strategy is an effective and affordable globally deployed Osprey fleet.

Page 151 GAO-11-233SP Assessments of Selected Weapon Programs Common Name: VXX Presidential Helicopter (VXX) The Navy’s VXX program is expected to develop and field a replacement for the current fleet of VH-3D and VH-60N helicopters that provide transportation for the President, Vice President, heads of state, and others as directed. The program was initiated after the VH-71 presidential helicopter replacement program was terminated in June 2009 due to excessive cost growth and schedule delays. The Navy is also taking steps to extend the service life of its existing fleet of VH-3D and VH-60N helicopters until a new helicopter can be fielded.

Source: U.S. Navy. Current Status

In March 2010, the VXX program held a materiel development decision review. The Navy expects to complete an analysis of alternatives by the second quarter of fiscal year 2011 and enter technology development by the third quarter of fiscal year 2011. According to program officials, the new program will complete a four-phase systems engineering and design review process before beginning system development.

In June 2009, DOD terminated the VH-71 program due to excessive cost growth, schedule delays, and technical risk. From its inception, program officials had acknowledged that the program carried substantial risk due to an aggressive schedule directed by the White House. Program officials further attribute the program’s problems to a misunderstanding between the government and contractors over requirements—a problem exacerbated by a highly compressed development schedule, a lack of sufficient predevelopment systems engineering and design work, and the eventual reengineering of entire subsystems needed to mitigate aircraft performance and weight issues. Prior to termination, five pilot production VH-71s were built and delivered and 10 percent of the planned 1,460 flight hour test program was completed.

As a result of the VH-71 termination, the Navy has to extend the service life of its fleet of VH-3D and VH-60N helicopters. Current projections indicate these helicopters can operate until 2017; however, the Navy is studying whether the fleet’s life can be extended to 2023 or later until the VXX is fielded.

Estimated Total Program Cost: TBD Quantities: TBD

Next Major Program Event: Technology development start, third quarter fiscal year 2011

Program Office Comments: The program office concurred with this assessment.

Page 152 GAO-11-233SP Assessments of Selected Weapon Programs

Agency Comments and DOD provided us with written comments on a draft of this report. The comments are reprinted in appendix VI. DOD also provided technical Our Evaluation comments, which we addressed in the report, as appropriate.

In its comments, DOD stated that it did not find GAO’s methods for calculating cost growth useful for management purposes. Specifically, DOD takes exception to the part of our analysis that aggregates data on the cost growth that has accumulated from a program’s first full cost estimate, which is typically tied to the start of development—milestone B—to the current year. DOD considers our methodology to be flawed and the resulting information to be misleading because it does not delineate cost growth experienced in the past, cost growth associated with capability upgrades, and cost growth associated with quantity increases. DOD requested that those metrics be removed from the report. In addition, DOD commented that the four different cost growth metrics presented in the report make it difficult for readers to gain an understanding of program performance.

Given the magnitude and complexity of major weapon system acquisitions, we believe no single metric adequately captures all of the dynamics of cost changes. Measuring cost increases from the formal start of a program to the present is one of several important metrics because it conveys the magnitude of the task at hand and provides a context for management. Further, milestone B is recognized as a key point for establishing accountability for weapon system programs, as evidenced by the importance of milestone B cost estimates in reporting Nunn-McCurdy unit cost breaches to Congress and the statutory certification requirements a program must meet, which include developing a reasonable cost estimate, to proceed beyond this point. Therefore, we retained this as one of several metrics used to measure program performance in this report. With regard to DOD’s comment about cost growth associated with capability upgrades, we believe that a key tenet of a knowledge-based acquisition approach is to set realistic capability requirements at the outset of a program and to avoid changing them in order to achieve cost and schedule predictability.

We used several other metrics to make distinctions between older and more recent cost increases and increases stemming from a lack of knowledge or poor program management versus simple quantity increases. For example, we designed our cost analysis to focus primarily on program performance over the last 2 years, which allows us to evaluate DOD’s management of its major defense acquisition programs since key

Page 153 GAO-11-233SP Assessments of Selected Weapon Programs

acquisition reforms were put into place by Congress and DOD. To ensure the focus remains on this 2-year analysis, we made changes to table 2 in the final report in response to DOD’s comments. Our analysis also explicitly accounts for cost increases associated with changes in weapon system quantities including cases such as the DDG 51 and the MRAP. Similarly, we present analysis of the often-overlooked and hard-to-quantify phenomenon of reducing quantities or capabilities, or both, to offset cost increases.

DOD also commented on a set of program performance metrics that were discussed by DOD, OMB, and GAO in 2008, as a mechanism for evaluating DOD’s progress in addressing the issues discussed in GAO’s Weapon Systems Acquisition High-Risk area. DOD believes that these metrics still do not adequately capture cost growth that results solely from poor estimating and poor execution as opposed to other sources including changes in inventory goals and changes in requirements or capabilities. We agree that these metrics do not quantify the cost growth attributable to each of those factors; however, DOD specifically requested that we use these metrics to assess the performance of its major defense acquisition program portfolio and its policies in its comments on our 2009 and 2010 annual assessments of weapon programs. We will continue to work with DOD to develop a set of metrics to better measure its progress in addressing its long-standing weapon system acquisition issues. Finally, DOD stated that it is undertaking a series of actions to obtain greater efficiency and productivity in defense spending. We support DOD’s efforts to get better value from defense spending and look forward to including the results of these efforts in future reports.

We are sending copies of this report to the Secretary of Defense; the Secretaries of the Army, Navy, and Air Force; and the Director of the Office of Management and Budget. In addition, the report will be made available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact me at (202) 512-4841. Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page

Page 154 GAO-11-233SP Assessments of Selected Weapon Programs

of this report. Staff members making key contributions to this report are listed in appendix VII.

Michael J. Sullivan Director, Acquisition and Sourcing Management

Page 155 GAO-11-233SP Assessments of Selected Weapon Programs

List of Committees

The Honorable Carl Levin Chairman The Honorable John McCain Ranking Member Committee on Armed Services United States Senate

The Honorable Daniel Inouye Chairman The Honorable Thad Cochran Ranking Member Subcommittee on Defense Committee on Appropriations United States Senate

The Honorable Howard P. McKeon Chairman The Honorable Adam Smith Ranking Member Committee on Armed Services House of Representatives

The Honorable C.W. Bill Young Chairman The Honorable Norman D. Dicks  Ranking Member Subcommittee on Defense Committee on Appropriations House of Representatives

Page 156 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix I

Scope and Methodology AppendixesAppendixI

This report contains observations on the performance of the Department of Defense’s (DOD) fiscal year 2010 major defense acquisition program portfolio. To develop these observations, we obtained and analyzed data from Selected Acquisition Reports (SAR) and other information in the Defense Acquisition Management Information Retrieval Purview system, 1 referred to as DAMIR. We refer to programs with SARs dated December 2009 as the 2010 portfolio. We converted cost information to fiscal year 2011 dollars using conversion factors from the DOD Comptroller’s National Defense Budget Estimates for Fiscal Year 2011 (table 5-9). Data for the total planned investment of major defense acquisition programs were obtained from DAMIR, which we aggregated for all programs using fiscal year 2011 dollars. However, the data do not include the full costs of acquiring Missile Defense Agency (MDA) programs.

We also collected and analyzed data on the composition of DOD’s major defense acquisition program portfolio. To determine changes in that portfolio, we compared the programs that issued SARs in December 2009 with the list of programs that issued SARs in December 2007. To assess the cost effect of changes to the major defense acquisition portfolio, we calculated the estimated total acquisition cost for the 13 programs exiting the portfolio and for the 15 programs entering the portfolio.

To compare the cost of major defense acquisition programs over 2 years, 5 years, and from baseline estimates, we collected data from December 2009, December 2007, and December 2004 SARs; acquisition program baselines; and program offices. We retrieved data that showed cost estimates for research, development, test, and evaluation; procurement; and total acquisition for 98 major defense acquisition programs in the 2010 portfolio. We divided some SAR programs into smaller elements, because DOD reports performance data on them separately. We analyzed the data to determine the change in research and development, procurement, and total acquisition costs from the first full estimate, generally development start, with the current estimate. For a few programs that did not have a development estimate, we compared the current estimate to the production estimate. Also, for a few shipbuilding programs that had a full planning estimate, we compared the current estimate to the planning estimate. For programs that began as non–major defense acquisition

1DAMIR Purview is an executive information system operated by the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics / Acquisition Resources and Analysis.

Page 157 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix I Scope and Methodology

programs, the first full estimate used by GAO as a baseline may be different than the original baseline contained in DOD SARs. When comparable cost and schedule data were not available for programs, we excluded them from the analysis. To calculate cost growth incurred over the past 2 years, from 2008 to 2010, we calculated the difference between the December 2007 and December 2009 SARs for programs older than 2 years. For programs less than 2 years old, we calculated the difference between December 2009 and first full estimates. We converted all dollar figures to fiscal year 2011 constant dollars. We took a similar approach for calculating cost growth incurred from 2005 to 2010. We also obtained schedule information and calculated the time since program start, or program age; cycle time from program start to initial operational capability; delay in obtaining initial operational capability; and the delay in initial capability as a percentage of total cycle time. Finally, we extracted data on quantities and program acquisition unit cost and compared the current unit cost to the baseline unit cost to determine whether programs’ unit cost has increased or decreased from the baseline estimate. For three software programs with no quantities, we assigned each program a quantity of one. For two programs, F-22 and DDG 51, we then calculated the effect on DOD’s buying power from each program by multiplying the change in program acquisition unit cost by the current planned quantities.

To calculate the amount of procurement cost growth attributable to quantity increases, we isolated the change in procurement costs and the change in procurement quantities for programs over the past 2 years. For those programs with change in procurement quantities, we calculated the amount attributable to quantity changes as the change in quantity multiplied by the average procurement unit cost for the program 2 years ago.

To evaluate program performance according to DOD, OMB, and GAO– developed metrics, we calculated how many programs had less than a 4 percent increase in total acquisition cost over the past 2 years, less than a 10 percent increase over the past 5 years, and less than a 15 percent increase from initial estimates using data from December 2009, December 2007, and December 2004 SARs; acquisition program baselines; and program offices. For programs that began as non–major defense acquisition programs, the first full estimate used by GAO as a baseline may be different than the original baseline contained in DOD SARs. We also identified 10 of the highest cost programs from the December 2009 SARs and calculated changes in total costs and program acquisition unit cost over the past 2 years. We excluded MDA’s Ballistic Missile Defense System

Page 158 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix I Scope and Methodology

from this analysis, because the program does not report baseline estimates or quantity information in its annual SAR.

To calculate the amount of cost growth incurred before and after production start, we identified 56 programs with December 2009 SARs that had production cost estimates. To determine the average age of these programs, we calculated the time between program start and December 2009 for each program. To analyze the cost growth before and after production, we compared development cost estimates, production cost estimates, and December 2009 cost estimates for both research and development and procurement costs and determined the difference between each estimate for each program. We then calculated the percent change between development and production estimates, and between production and current estimates for both development and procurement costs. Finally, we calculated the average percent change for each program from development to production and from production to current estimates to determine the average percent cost growth incurred before and after production.

Through discussions with DOD officials responsible for the database and confirming selected data with program offices, we determined that the SAR data and the information retrieved from DAMIR were sufficiently reliable for our purposes.

Analysis of Selected In total, this report presents information on 71 weapon programs. A table listing these programs is found in appendix VII. Out of these programs, 49 DOD Programs Using are captured in a two-page format discussing technology, design, and Knowledge-Based manufacturing knowledge obtained and other program issues. The Criteria remaining 22 programs are described in a one-page format that describes their current status. We chose these programs based on their estimated cost, stage in the acquisition process, and congressional interest. To obtain cost, schedule, technology, design, and manufacturing information, we asked 49 programs to complete a data-collection instrument and received responses from all these programs. In addition, to collect information from major defense acquisition programs and components of these programs on other program factors such as requirements changes, configuration steering board activities, software development, and program office staffing, we asked 44 programs to complete a second electronic questionnaire and received responses from all these programs from June to November 2010. To collect data from pre–major defense acquisition programs including cost and schedule estimates, technology maturity, and

Page 159 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix I Scope and Methodology

planned implementation of acquisition reforms, we distributed a separate electronic questionnaire to 54 programs categorized as pre–major defense acquisition programs as of July 2010. Both questionnaires were sent by e- mail in an attached Microsoft Word form that respondents could return electronically. We received responses from June to November 2010. During the course of our review, we dropped 37 programs from this analysis, including 19 that were no longer slated to become major defense acquisition programs, 11 that had not yet begun the technology development phase, and 7 that had become major defense acquisition programs or were spinoffs of major defense acquisition programs. In addition, three programs did not return the questionnaire. Therefore, our assessment of planned major defense acquisition programs consists of 14 programs nearing system development start. To ensure the reliability of the data collected through our questionnaires, we took a number of steps to reduce measurement error, nonresponse error, and respondent bias. These steps included conducting two pretests of each questionnaire by phone prior to distribution to ensure that our questions were clear, unbiased, and consistently interpreted; reviewing responses to identify obvious errors or inconsistencies; conducting follow-up to clarify responses when needed; and verifying the accuracy of a sample of keypunched questionnaires.

Our analysis of how well programs are adhering to a knowledge-based acquisition approach focuses on a subset of 40 major defense acquisition programs from DOD’s fiscal year 2010 portfolio that were in development or the early stages of production as of June 2010. The 31 programs that are not included in this analysis either do not have acquisition milestones that line up with development start, critical design review, and production start or lack key data on technology, design, and production necessary to assess them against our knowledge-based acquisition criteria at this point in time.2

To assess the cost and schedule outcomes for the 40 programs to-date, we identified programs with cost, schedule, and quantity data at the first full estimate, generally milestone B—development start—and the estimate from the December 2009 SAR. Of the programs in our assessment, 39 had relevant data on research and development costs, 38 had relevant data on procurement costs, and 34 had data on schedules for delivering initial

2The 31 programs in our assessment that are not covered in this analysis include: 18 planned major defense acquisition programs, 4 MDA elements, 4 programs that are well into production, 1 component within a major defense acquisition program, 1 program that is based on a commercially-derived aircraft, 2 programs that were canceled, and 1 technology development program.

Page 160 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix I Scope and Methodology

capabilities. The remaining programs, not included in this analysis, did not have comparable data. We summed the first full estimate and the current estimate of both research and development costs and procurement costs for the programs and calculated the percentage change between the estimates. The schedule assessment is calculated two ways—as the average of the change in months between the first and current estimates for the planned or actual delivery of initial operational capability and as the average change in months divided by the first estimate of acquisition cycle time.

To assess knowledge attainment of programs at critical decision points (system development start, critical design review, and production start), we collected data about their knowledge levels at each point. The data were collected from 40 program offices as of November 2010. Additional information on product knowledge is found in the product knowledge assessment section of this appendix. We did not validate the data provided by the program offices, but reviewed the data and performed various checks to determine that they were reliable enough for our purposes. Where we discovered discrepancies, we clarified the data accordingly. Programs in our assessment were in various stages of the acquisition cycle, and not all of the programs provided knowledge information for each point. Programs were not included in our assessments if relevant decision or knowledge point data were not available. In addition, because knowledge points differ for shipbuilding programs, we exclude them from our assessment of certain knowledge-based practices. In particular, we focused on the 17 programs that entered these key acquisition points since 2009 and evaluated their adherence to knowledge-based practices. For each decision point, we summarized knowledge attainment for the number of programs with data that achieved that knowledge point. Twenty-six nonship programs provided data on technology maturity at development start, 1 of which began development since 2009; 29 nonship programs provided data on design stability at their critical design review, 9 of which held this review since 2009; and 4 programs provided data on production processes in control at production start, 1 of which began production since 2009. Our analysis of knowledge attained at each key point also includes other factors that we have previously identified as being key to a knowledge-based acquisition approach, including holding system design reviews early in development, planning for manufacturing, testing an integrated prototype prior to the design review, using a reliability growth curve, and testing a production-representative prototype prior to making a production decision. See appendix IV for a list of these practices.

Page 161 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix I Scope and Methodology

For our analysis of requirements changes, we obtained and analyzed information from 39 programs about the number and effect of requirements changes since development start. Using this information, we compared the average percent change in research and development cost and delay in delivery of an initial operational capability between programs that had an added or enhanced key performance parameter; a reduced, deferred, or deleted key performance parameter; or stable key performance parameters. We also compared the age of programs that had stable or unstable requirements.

For our analysis of software development, we obtained and analyzed information from 25 programs related to the number of software lines of code expected in the final system at development start and currently, and from 21 programs that collect data on the percentages of software defects contained in-phase and in subsequent phases. We also collected data from 40 programs on whether they collected earned value management data for software development. We calculated whether the percentage growth in total lines of code correlated with the percentage growth in research and development costs, as well as whether the growth in software correlated with percentage delay in achieving initial operational capabilities. We also compared the total lines of code expected currently with whether programs are collecting earned value management data for software development. Finally we compared the percentage growth in software lines of code with program age.

For our analysis of program staffing, we analyzed information related to program office staffing from 44 programs, including 40 major defense acquisition programs and 4 MDA elements, on the number of military personnel, civilian government employees, support contractors, and Federally Funded Research and Development Centers and university- affiliated employees working in the following functions: program management, business-related functions, contracting, engineering, administrative support, and other functions. We compared this information with data collected in prior years. We also collected information on whether programs had been authorized all positions requested, whether they had filled those positions, reasons for not filling positions, and whether they were using support contractors to make up for shortfalls in government personnel or capabilities.

To determine how DOD has begun to implement acquisition reforms, we obtained and analyzed the revised DOD 5000.02 acquisition instruction, the Weapon Systems Acquisition Reform Act of 2009, and the Directive-Type

Page 162 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix I Scope and Methodology

Memorandum 09-027 implementing the Act. We analyzed data from the survey sent to planned major defense acquisition programs in our assessment to determine how they were implementing requirements for holding a preliminary design review; developing prototypes; maturing critical technologies; and considering tradeoffs among cost, schedule, and performance objectives before development start. We also collected information on whether these programs are planning to incorporate competition into their acquisition strategies.3 To determine how programs were implementing the requirement to hold configuration steering boards, we analyzed data from the survey sent to major defense acquisition programs.

We relied on GAO’s body of work examining DOD acquisition issues over the years. In recent years, we have issued reports that identified systemic problems with major weapon systems acquisitions and we have made recommendations to DOD on ways to improve how it acquires major weapon systems. These reports cover contracting, program management, acquisition policy, cost estimating, budgeting, and requirements development. We have also issued many detailed reports evaluating specific weapon systems, such as aircraft programs, ships, communication systems, satellites, missile defense systems, and future combat systems. We also used information from numerous GAO products that examine how commercial best practices can improve outcomes for DOD programs. This work has shown that valuable lessons can be learned from the commercial sector and can be applied to the development of weapon systems.

System Profile Data on Over the past several years, DOD has revised policies governing weapon system acquisitions and changed the terminology used for major Each Individual Two- acquisition events. To make DOD’s acquisition terminology more consistent Page Assessment across the 71 program assessments, we standardized the terminology for

3The DOD and statutory requirement is that the acquisition strategy for each major defense acquisition program include measures to ensure competition, or the option of competition, throughout the life cycle of the program. Weapon Systems Acquisition Reform Act of 2009, Pub. L. No. 111-23, § 202; Under Secretary of Defense, Acquisition, Technology and Logistics Directive-Type Memorandum (DTM) 09-027—Implementation of the Weapon Systems Acquisition Reform Act of 2009, attachment 1, para. 2 (Dec. 4, 2009). The survey question with respect to this requirement read “Does the acquisition strategy call for competition post–Milestone B.” When programs answered “no” to the question, GAO interpreted that answer to mean that the program is not planning to incorporate into the acquisition strategy competition, or the option of competition, after development start.

Page 163 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix I Scope and Methodology

key program events. For most individual programs in our assessment, “development start” refers to the initiation of an acquisition program as well as the start of engineering and manufacturing development. This coincides with DOD’s milestone B. A few programs in our assessment (mostly programs that began before 2001) have a separate “program start” date, which begins a pre–system development phase for program definition and risk-reduction activities. This “program start” date generally coincides with DOD’s former terminology for milestone I, followed by a “development start” date, either DOD’s former milestone II or current milestone B depending on when the program began system development. The “production decision” generally refers to the decision to enter the production and deployment phase, typically with low-rate initial production. The “initial capability” refers to the initial operational capability—sometimes called first unit equipped of required asset availability. For shipbuilding programs, the schedule of key program events in relation to acquisition milestones varies for each program. Our work on shipbuilding best practices has identified the detailed design and construction contract award and the start of lead ship fabrication as the points in the acquisition process roughly equivalent to development start and design review for other programs. For MDA programs that do not follow the standard DOD acquisition model but instead develop systems’ capabilities incrementally, we identify the key technology development efforts that lead to an initial capability.

For each program we assessed in a two-page format, we present cost, schedule, and quantity data at the program’s first full estimate, generally milestone B, and an estimate from the program office reflecting 2010 data where it was available. To assess the cost, schedule, and quantity changes of each program, we reviewed DOD’s SARs or obtained data directly from the program offices. In general, we compared the latest available SAR information with a baseline for each program. For programs that have started product development—those that are beyond milestone II or B—we compared the latest available SAR to the development estimate from the first SAR issued after the program was approved to enter development, or for the planning estimate if we had a full estimate. For systems not included in the SARs, we attempted to obtain comparable baseline and current data from the individual program offices. For MDA systems, for which a baseline was not available, we do not present a comparison. For the other programs assessed in a one-page format, we present the latest available estimate of cost and quantity from the program office.

Page 164 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix I Scope and Methodology

For each program we assessed, all cost information is presented in fiscal year 2011 dollars using Office of the Secretary of Defense–approved deflators to eliminate the effects of inflation. We have depicted only the program’s main elements of acquisition cost—research and development and procurement. However, the total program cost also includes military construction and acquisition operation and maintenance costs. Because of rounding and these additional costs, in some situations, total cost may not match the exact sum of the research and development and procurement costs. The program unit costs are calculated by dividing the total program cost by the total quantities planned. In some instances, the data were not applicable, and we annotate this by using the term “not applicable (NA).” The quantities listed refer to total quantities, including both procurement and development quantities.

The schedule assessment for each program is based on acquisition cycle time, defined as the number of months between program start and the achievement of initial operational capability or an equivalent fielding date. In some instances the data were not yet available, and we annotate this by using the term “to be determinded (TBD)” or “NA.”

The information presented on the “funding needed to complete” is from fiscal year 2011 through completion and, unless otherwise noted, draws on information from SARs or on data from the program office. In some instances, the data were not available, and we annotate this by the term “TBD” or “NA.” The quantities listed refer only to procurement quantities. Satellite programs, in particular, produce a large percentage of their total operations units as development quantities, which are not included in the quantity figure.

The intent of these comparisons is to provide an aggregate, or overall, picture of a program’s history. These assessments represent the sum of the federal government’s actions on a program, not just those of the program manager and the contractor. DOD does a number of detailed analyses of changes that attempt to link specific changes with triggering events or causes. Our analysis does not attempt to make such detailed distinctions.

Product Knowledge In our past work examining weapon acquisition issues and best practices for product development, we have found that leading commercial firms Data on Individual pursue an acquisition approach that is anchored in knowledge, whereby Two-Page Assessments high levels of product knowledge are demonstrated by critical points in the acquisition process. On the basis of this work, we have identified three key

Page 165 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix I Scope and Methodology

knowledge points during the acquisition cycle—development start; design review, which occurs during engineering and manufacturing development; and production start—at which programs need to demonstrate critical levels of knowledge to proceed. To assess the product development knowledge of each program at these key points, we submitted a data- collection instrument to 49 program offices. We received responses from all 49 programs; however, not every program had responses to each element of the data-collection instrument. The results are graphically depicted in each two-page assessment. We also reviewed pertinent program documentation and discussed the information presented on the data-collection instrument with program officials as necessary.

To assess technology maturity, we asked program officials to apply a tool, referred to as Technology Readiness Levels (TRL), for our analysis. The National Aeronautics and Space Administration originally developed TRLs, and the Army and Air Force science and technology research organizations use them to determine when technologies are ready to be handed off from science and technology managers to product developers. TRLs are measured on a scale from 1 to 9, beginning with paper studies of a technology’s feasibility and culminating with a technology fully integrated into a completed product. See appendix V for TRL definitions. Our best practices work has shown that a technology readiness level of 7— demonstration of a technology in a realistic environment—is the level of technology maturity that constitutes a low risk for starting a product development program.4 For shipbuilding programs, we have recommended that this level of maturity be achieved by the contract award for detailed design and construction.5 In our assessment, the technologies that have reached TRL 7, a prototype demonstrated in a realistic environment, are referred to as mature or fully mature. Those technologies that have reached TRL 6, a prototype demonstrated in a relevant environment, are referred to as approaching or nearing maturity and are assessed at attaining 50 percent of the desired level of knowledge. Satellite technologies that have achieved TRL 6 are assessed as fully mature due to the difficulty of demonstrating maturity in a realistic environment—space.

4GAO, Best Practices: Better Management of Technology Development Can Improve Weapon System Outcomes, GAO/NSIAD-99-162 (Washington, D.C.: July 30, 1999); Best Practices: Better Matching of Needs and Resources Will Lead to Better Weapon System Outcomes, GAO-01-288 (Washington, D.C.: Mar. 8, 2001).

5GAO, Best Practices: High Levels of Knowledge at Key Points Differentiate Commercial Shipbuilding from Navy Shipbuilding, GAO-09-322 (Washington, D.C.: May 13, 2009).

Page 166 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix I Scope and Methodology

In most cases, we did not validate the program offices’ selection of critical technologies or the determination of the demonstrated level of maturity. We sought to clarify the TRLs in those cases where information existed that raised concerns. If we were to conduct a detailed review, we might adjust the critical technologies assessed, their readiness levels demonstrated, or both. It was not always possible to reconstruct the technological maturity of a weapon system at key decision points after the passage of many years. Where practicable, we compared technology assessments provided by the program office to assessments conducted by officials from the Office of the Director, Defense Research and Engineering.

To assess design stability, we asked program officials to provide the percentage of engineering drawings completed or projected for completion by the design review, the production decision, and as of our current assessment.6 In most cases, we did not verify or validate the percentage of engineering drawings provided by the program office. We clarified the percentage of drawings completed in those cases where information that raised concerns existed. Completed drawings were defined as the number of drawings released or deemed releasable to manufacturing that can be considered the “build to” drawings. For shipbuilding programs, we asked program officials to provide the percentage of the 3D product model that had been completed by the start of lead ship fabrication, and as of our current assessment.7

To assess production maturity, we asked program officials to identify the number of critical manufacturing processes and, where available, to quantify the extent of statistical control achieved for those processes.8 In most cases, we did not verify or validate the information provided by the program office. We clarified the number of critical manufacturing processes and the percentage of statistical process control where information existed that raised concerns. We used a standard called the Process Capability Index, a process performance measurement that quantifies how closely a process is running to its specification limits. The index can be translated into an expected product defect rate, and we have found it to be a best practice. We sought other data, such as scrap and

6GAO, Best Practices: Capturing Design and Manufacturing Knowledge Early Improves Acquisition Outcomes, GAO-02-701 (Washington, D.C.: July 15, 2002).

7GAO-09-322.

8GAO-02-701.

Page 167 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix I Scope and Methodology

rework trends in those cases where quantifiable statistical control data were unavailable. We do not assess production maturity for shipbuilding programs.

Although the knowledge points provide excellent indicators of potential risks, by themselves they do not cover all elements of risk that a program encounters during development, such as funding instability. Our detailed reviews on individual systems normally provide a more comprehensive assessment of risk elements.

We conducted this performance audit from June 2010 to March 2011, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Page 168 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix II Changes in DOD’s 2010 Portfolio of Major

Defense Acquisition Programs over Time AppendixII

Table 4 shows the change in research and development costs, procurement costs, total acquisition cost, and average delay in delivering initial operational capability for the Department of Defense’s (DOD) 2010 portfolio of major defense acquisition programs. The table presents changes that have occurred on these programs in the last 2 years, the last 5 years, and since their first full cost and schedule estimates.

Table 4: Changes in DOD’s 2010 Portfolio of Major Defense Acquisition Programs over Time

Fiscal year 2011 dollars in billions Last 2 years Last 5 years Since first full estimate (2008 to 2010) (2005 to 2010) (Baseline to 2010) Increase in total research and development cost $15 $29 $102 5 percent 10 percent 47 percent Increase in total procurement cost $121 $186 $287 11 percent 18 percent 31 percent Increase in total acquisition cost $135 $217 $402 9 percent 16 percent 35 percent Average delay in delivering initial capabilities 5 months 9 months 22 months 8 percent 13 percent 30 percent Source: GAO analysis of DOD data. Notes: Data were obtained from DOD’s Selected Acquisition Reports. In a few cases data were obtained directly from program offices. Not all programs had comparable cost and schedule data and these programs were excluded from the analysis where appropriate. Portfolio performance data do not include costs of developing Missile Defense Agency elements. Total acquisition cost includes research and development, procurement, acquisition operation and maintenance, and system-specific military construction costs.

Page 169 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix III Current and Baseline Cost Estimates for DOD’s 2010 Portfolio of Major Defense

Acquisition Programs AppendixIII

Table 5 contains the current and baseline total acquisition cost estimates (in fiscal year 2011 dollars) for each program or element in the Department of Defense’s (DOD) 2010 major defense acquisition program portfolio. We excluded elements of the Missile Defense Agency’s Ballistic Missile Defense System because comparable current and baseline cost estimates were not available. For each program we show the percent change in total acquisition cost from the program baseline, as well as over the past 2 years and 5 years.

Table 5: Current Cost Estimates and Baseline Cost Estimates for DOD’s 2010 Portfolio of Major Defense Acquisition Programs

Fiscal year 2011 dollars in millions Change in Change in total Change in total total acquisition acquisition Current total Baseline total acquisition cost within the cost within the acquisition acquisition cost from past 2 years past 5 years Program cost cost baseline (%) (%) (%) Advanced Extremely High Frequency (AEHF) $12,920 $6,277 105.8 61.3 87.1 Satellite Advanced Threat Infrared 4,701 3,362 39.8 -4.2 -1.6 Countermeasure/Common Missile Warning System (ATIRCM/CMWS) AGM-88E Advanced Anti-Radiation Guided 1,825 1,577 15.7 8.7 11.2 Missile (AARGM) AH-64D Longbow Apache 14,507 6,041 140.2 15.4 38.9 AIM-120 Advanced Medium Range Air-to-Air 23,900 10,767 122.0 30.8 41.7 Missile (AMRAAM) AIM-9X/Air-to-Air Missile 3,589 3,096 15.9 7.2 16.2 Airborne Signals Intelligence Payload (ASIP)– 546 342 59.5 NA NA Baseline Apache Block III (AB3) 10,577 7,135 48.2 35.7 48.2 Army Integrated Air & Missile Defense (Army 4,954 4,954 0.0 0.0 0.0 IAMD) B-2 Extremely High Frequency (EHF) SATCOM 619 699 -11.6 -8.3 -11.6 Capability, Increment 1 B-2 Radar Modernization Program (B-2 RMP) 1,305 1,319 -1.1 3.4 2.8 Black Hawk (UH-60M) 21,936 12,779 71.7 3.2 16.3 Block IV Tomahawk (Tactical Tomahawk) 6,845 2,085 228.3 49.2 50.9 Bradley Fighting Vehicle Systems (BFVS) A3 9,670 4,119 134.8 -5.8 200.4 Upgrade Broad Area Maritime Surveillance (BAMS) 13,032 12,657 3.0 3.0 3.0 Unmanned Aircraft System (UAS)

Page 170 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix III Current and Baseline Cost Estimates for DOD’s 2010 Portfolio of Major Defense Acquisition Programs

(Continued From Previous Page) Fiscal year 2011 dollars in millions Change in Change in total Change in total total acquisition acquisition Current total Baseline total acquisition cost within the cost within the acquisition acquisition cost from past 2 years past 5 years Program cost cost baseline (%) (%) (%) C-130 Avionics Modernization Program (C-130 6,053 4,071 48.7 9.6 33.3 AMP) C-130J Hercules 15,327 935 1,539.3 22.2 117.4 C-17A Globemaster III 82,347 52,769 56.0 9.7 13.9 C-27J Joint Cargo Aircraft (JCA) 1,966 3,854 -49.0 -49.0 -49.0 C-5 Avionics Modernization Program (C-5 AMP) 1,320 1,087 21.5 -11.9 38.3 C-5 Reliability Enhancement and Reengining 7,348 10,744 -31.6 -29.0 -30.4 Program (C-5 RERP) CH-47F Improved Cargo Helicopter (CH-47F) 13,530 3,172 326.6 4.7 13.5 CH-53K—Heavy Lift Replacement 21,902 16,311 34.3 33.9 34.3 Chemical Demilitarization–Assembled Chemical 7,935 2,596 205.6 8.6 73.0 Weapons Alternatives (Chem Demil-ACWA) Chemical Demilitarization–Chemical Materials 28,362 15,260 85.9 -4.7 0.1 Agency (Chem Demil-CMA) Cobra Judy Replacement (CJR) 1,797 1,607 11.8 5.0 12.4 Cooperative Engagement Capability (CEC) 5,063 2,897 74.8 1.2 -0.1 CVN 21 Future Aircraft Carrier 34,186 35,048 -2.5 12.0 2.9 CVN-68 Class / Carrier Replacement Program 6,873 5,948 15.6 -0.2 -3.5 (CVN 77) DDG 1000 Destroyer 19,810 34,284 -42.2 -29.7 120.6 DDG 51 Destroyer 94,344 14,960 530.6 21.9 20.8 E-2D Advanced Hawkeye (E-2D AHE) 17,831 14,535 22.7 12.0 21.0 EA-18G Growler 11,601 8,843 31.2 32.6 32.3 EA-6B Improved Capability (ICAP) III 1,187 1,170 1.4 1.4 1.4 Excalibur Precision Guided Extended Range 2,437 4,706 -48.2 3.0 10.3 Artillery Projectile Expeditionary Fighting Vehicle (EFV) 14,044 9,019 55.7 0.6 14.6 Gray Eagle 4,978 1,000 397.7 108.6 397.7 F/A-18E/F Super Hornet 54,625 80,513 -32.2 3.4 7.7 F-22 Raptor 77,393 89,901 -13.9 2.9 6.8 F-35 Lightning II (Joint Strike Fighter) 283,674 210,558 34.7 13.6 23.9 Family of Advanced Beyond Line-of-Sight 3,930 3,141 25.1 11.6 25.1 Terminals (FAB-T) Family of Medium Tactical Vehicles (FMTV) 21,301 10,292 107.0 0.8 19.6 Force XXI Battle Command Brigade and Below 4,113 2,785 47.7 13.8 102.3 (FBCB2)

Page 171 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix III Current and Baseline Cost Estimates for DOD’s 2010 Portfolio of Major Defense Acquisition Programs

(Continued From Previous Page) Fiscal year 2011 dollars in millions Change in Change in total Change in total total acquisition acquisition Current total Baseline total acquisition cost within the cost within the acquisition acquisition cost from past 2 years past 5 years Program cost cost baseline (%) (%) (%) Global Broadcast Service (GBS) 1,118 567 97.4 22.5 28.0 Global Hawk (RQ-4A/B) 13,576 5,312 155.5 37.2 92.7 Guided Multiple Launch Rocket System (GMLRS) 5,767 1,742 231.1 3.8 -53.8 H-1 Upgrades (UH-1Y/AH-1Z) 11,866 3,572 232.2 37.4 47.3 High Mobility Artillery Rocket System (HIMARS) 2,126 4,297 -50.5 0.5 -52.3 Increment 1 Early-Infantry Brigade Combat Team 3,077 3,184 -3.4 -3.4 -3.4 (E-IBCT) Integrated Defensive Electronic Countermeasures 692 684 1.2 1.2 1.2 (IDECM) Block 4 Integrated Defensive Electronic Countermeasures 1,531 1,461 4.8 4.8 4.8 (IDECM) Blocks 2/3 Joint Air-to-Surface Standoff Missile (JASSM) 7,201 2,282 215.6 23.6 50.5 Joint Direct Attack Munition (JDAM) 6,377 3,367 89.4 8.1 3.0 Joint High Speed Vessel (JHSV) 3,669 3,583 2.4 2.4 2.4 Joint Land Attack Cruise Missile Defense Elevated 7,378 6,567 12.4 7.9 12.4 Netted Sensor System (JLENS) Joint Mine Resistant Ambush Protected (MRAP) 36,375 22,792 59.6 59.6 59.6 Joint Precision Approach and Landing System 971 997 -2.6 -2.6 -2.6 (JPALS) Joint Primary Aircraft Training System (JPATS) 5,815 3,670 58.4 -0.5 2.5 Joint Standoff Weapon (JSOW) Baseline 2,205 2,813 -21.6 0.2 -0.6 Joint Standoff Weapon (JSOW) Unitary 3,125 5,015 -37.7 17.9 9.8 Joint Tactical Radio System (JTRS) Ground 15,868 17,165 -7.6 -6.3 -11.8 Mobile Radios (GMR) Joint Tactical Radio System (JTRS) Handheld, 4,786 9,889 -51.6 55.2 -51.6 Manpack, and Small Form Fit (HMS) Joint Tactical Radio System (JTRS) Network 1,995 966 106.4 -3.8 43.1 Enterprise Domain (NED) Airborne and Maritime/Fixed Station Joint Tactical 8,212 8,033 2.2 2.2 2.2 Radio System (AMF JTRS) Large Aircraft Infrared Countermeasures 454 397 14.4 14.4 14.4 (LAIRCM) Lewis and Clark Class (T-AKE) Dry 6,586 5,205 26.5 16.8 35.7 Cargo/Ammunition Ship LHA Replacement Amphibious Assault Ship 6,387 3,133 103.9 90.5 103.9 Light Utility Helicopter (LUH), UH-72A Lakota 1,969 1,784 10.4 -0.6 10.4

Page 172 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix III Current and Baseline Cost Estimates for DOD’s 2010 Portfolio of Major Defense Acquisition Programs

(Continued From Previous Page) Fiscal year 2011 dollars in millions Change in Change in total Change in total total acquisition acquisition Current total Baseline total acquisition cost within the cost within the acquisition acquisition cost from past 2 years past 5 years Program cost cost baseline (%) (%) (%) Littoral Combat Ship (LCS) 3,865 1,353 185.6 29.1 165.9 LPD 17 Amphibious Transport Dock 18,361 11,539 59.1 24.7 39.3 MH-60R Multi-Mission Helicopter 14,340 5,453 163.0 16.8 23.5 MH-60S Fleet Combat Support Helicopter 8,318 3,456 140.7 2.5 1.4 Minuteman III Propulsion Replacement Program 2,913 2,775 5.0 0.1 1.1 (PRP) Mobile User Objective System (MUOS) 6,830 6,622 3.1 4.4 14.6 Multifunctional Information Distribution System 2,939 1,284 129.0 9.1 21.0 (MIDS) Multi-Platform Radar Technology Insertion 1,363 1,770 -23.0 0.1 -18.7 Program (MP-RTIP) National Airspace System (NAS) 1,597 855 86.8 0.0 -1.2 National Polar-orbiting Operational Environmental 6,309 6,584 -4.2 -43.3 -13.9 Satellite System (NPOESS) Navstar Global Positioning System (GPS) IIIA 4,141 3,883 6.6 6.6 6.6 Navstar Global Positioning System (GPS) Space & 7,361 6,125 20.2 1.1 0.0 Control Navstar Global Positioning System (GPS) User 2,165 974 122.2 -1.0 49.6 Equipment Navy Multiband Terminal (NMT) 2,005 2,287 -12.3 0.6 -12.3 P-8A Poseidon 32,361 30,576 5.8 7.1 7.1 PATRIOT Advanced Capability-3 (PAC-3) 10,768 5,136 109.7 8.0 7.2 PATRIOT/Medium Extended Air Defense System 18,513 19,077 -3.0 1.5 -0.4 (MEADS) Combined Aggregate Program (CAP) Fire Unit PATRIOT/Medium Extended Air Defense System 7,677 7,179 6.9 10.4 9.2 (MEADS) Combined Aggregate Program (CAP) Missile Predator—Unmanned Aircraft System 3,581 3,579 0.1 0.1 0.1 Reaper Unmanned Aircraft System 11,132 2,598 328.5 328.5 328.5 Remote Minehunting System (RMS) 1,275 1,420 -10.2 -15.1 -10.2 Sea-Launched Ballistic Missile-UGM 133A Trident 51,410 50,942 0.9 1.6 4.4 II (D-5) Missile Space Based Infrared System (SBIRS) High 15,938 4,521 252.6 27.9 48.6 Program Space Based Space Surveillance (SBSS) Block 922 859 7.3 7.3 7.3 10

Page 173 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix III Current and Baseline Cost Estimates for DOD’s 2010 Portfolio of Major Defense Acquisition Programs

(Continued From Previous Page) Fiscal year 2011 dollars in millions Change in Change in total Change in total total acquisition acquisition Current total Baseline total acquisition cost within the cost within the acquisition acquisition cost from past 2 years past 5 years Program cost cost baseline (%) (%) (%) Standard Missile-6 (SM-6) Extended Range Active 6,133 5,616 9.2 13.2 12.0 Missile (ERAM) Stryker Family of Vehicles (Stryker) 16,153 7,914 104.1 -1.8 42.7 V-22 Joint Services Advanced Vertical Lift Aircraft 56,061 39,501 41.9 -1.1 3.3 (Osprey) Vertical Take-off and Landing Tactical Unmanned 2,469 2,576 -4.2 20.9 -4.2 Aerial Vehicle (VTUAV) Virginia Class Submarine (SSN 774) 82,193 59,550 38.0 -1.2 -7.8 Warfighter Information Network-Tactical (WIN-T) 4,738 3,653 29.7 29.7 29.7 Increment 2 Warfighter Information Network-Tactical (WIN-T) 13,552 16,125 -16.0 -16.0 -16.0 Increment 3 Warfighter Information Network-Tactical (WIN-T), 4,006 4,027 -0.5 -0.5 -0.5 Increment I Wideband Global SATCOM (WGS) $3,561 $1,175 203.1 68.4 75.6 Source: GAO analysis of DOD data. Notes: Data were obtained from DOD’s SARs, acquisition program baselines, and, in some cases, program offices. NA indicates data were not available to make the assessment.

Page 174 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix IV

Knowledge-Based Acquisition Practices AppendixIV

GAO’s prior work on best product development practices found that successful programs take steps to gather knowledge that confirms that their technologies are mature, their designs stable, and their production processes are in control. Successful product developers ensure a high level of knowledge is achieved at key junctures in development. We characterize these junctures as knowledge points. The Related GAO Products section of this report includes references to the body of work that helped us identify these practices and apply them as criteria in weapon system reviews. The following summarizes these knowledge points and associated key practices.

Knowledge Point 1: Technologies, time, funding, and other resources match customer needs. Decision to invest in product development Demonstrate technologies to a high readiness level—technology readiness level 7—to ensure technologies will work in an operational environment Ensure that requirements for product increment are informed by preliminary design review using systems engineering process (such as prototyping of preliminary design) Establish cost and schedule estimates for product on the basis of knowledge from preliminary design using system engineering tools (such as prototyping of preliminary design) Constrain development phase (5 to 6 years or less) for incremental development Ensure development phase fully funded (programmed in anticipation of milestone) Align program manager tenure to complete development phase Contract strategy that separates system integration and system demonstration activities Conduct independent cost estimate Conduct independent program assessment Conduct major milestone decision review for development start Knowledge Point 2: Design is stable and performs as expected. Decision to start building and testing production- representative prototypes Complete system critical design review Complete 90 percent of engineering design drawing packages Complete subsystem and system design reviews Demonstrate with system-level integrated prototype that design meets requirements Complete the failure modes and effects analysis Identify key system characteristics Identify critical manufacturing processes Establish reliability targets and growth plan on the basis of demonstrated reliability rates of components and subsystems Conduct independent cost estimate Conduct independent program assessment Conduct major milestone decision review to enter system demonstration

Page 175 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix IV Knowledge-Based Acquisition Practices

(Continued From Previous Page) Knowledge Point 3: Production meets cost, schedule and quality targets. Decision to produce first units for customer Demonstrate manufacturing processes Build and test production-representative prototypes to demonstrate product in intended environment Test production-representative prototypes to achieve reliability goal Collect statistical process control data Demonstrate that critical processes are capable and in statistical control Conduct independent cost estimate Conduct independent program assessment Conduct major milestone decision review to begin production Source: GAO.

Page 176 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix V

Technology Readiness Levels AppendixV

Technology readiness level Description Hardware/software Demonstration environment 1. Basic principles observed Lowest level of technology None (paper studies and None and reported readiness. Scientific research analysis) begins to be translated into applied research and development. Examples might include paper studies of a technology’s basic properties 2. Technology concept and/or Invention begins. Once basic None (paper studies and None application formulated principles are observed, analysis) practical applications can be invented. The application is speculative and there is no proof or detailed analysis to support the assumption. Examples are still limited to paper studies. 3. Analytical and experimental Active research and Analytical studies and Lab critical function and/or development is initiated. This demonstration of nonscale characteristic proof of includes analytical studies and individual components (pieces of concept laboratory studies to physically subsystem) validate analytical predictions of separate elements of the technology. Examples include components that are not yet integrated or representative. 4. Component and/or Basic technological components Low-fidelity breadboard. Lab breadboard validation in are integrated to establish that Integration of nonscale laboratory environment the pieces will work together. components to show pieces will This is relatively “low fidelity” work together. Not fully compared to the eventual functional or form or fit but system. Examples include representative of technically integration of “ad hoc” hardware feasible approach suitable for in a laboratory. flight articles. 5. Component and/or Fidelity of breadboard High-fidelity breadboard. Lab demonstrating functionality breadboard validation in technology increases Functionally equivalent but not but not form and fit. May include relevant environment significantly. The basic necessarily form and/or fit (size flight demonstrating breadboard technological components are weight, materials, etc). Should in surrogate aircraft. Technology integrated with reasonably be approaching appropriate ready for detailed design studies. realistic supporting elements so scale. May include integration of that the technology can be several components with tested in a simulated reasonably realistic support environment. Examples include elements/subsystems to “high fidelity” laboratory demonstrate functionality. integration of components.

Page 177 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix V Technology Readiness Levels

(Continued From Previous Page) 6. System/subsystem model or Representative model or Prototype. Should be very close High-fidelity lab demonstration prototype demonstration in prototype system, which is well to form, fit and function. Probably or limited/restricted flight a relevant environment beyond the breadboard tested includes the integration of many demonstration for a relevant for TRL 5, is tested in a relevant new components and realistic environment. Integration of environment. Represents a supporting technology is well defined. major step up in a technology’s elements/subsystems if needed demonstrated readiness. to demonstrate full functionality Examples include testing a of the subsystem. prototype in a high fidelity laboratory environment or in simulated realistic environment. 7. System prototype Prototype near or at planned Prototype. Should be form, fit Flight demonstration in demonstration in a realistic operational system. Represents and function integrated with representative realistic environment a major step up from TRL 6, other key supporting environment such as flying test requiring the demonstration of elements/subsystems to bed or demonstrator aircraft. an actual system prototype in a demonstrate full functionality of Technology is well substantiated realistic environment, such as in subsystem. with test data. an aircraft, vehicle or space. Examples include testing the prototype in a test bed aircraft. 8. Actual system completed Technology has been proven to Flight-qualified hardware Developmental Test and and “flight qualified” through work in its final form and under Evaluation (DT&E) in the actual test and demonstration expected conditions. In almost system application. all cases, this TRL represents the end of true system development. Examples include developmental test and evaluation of the system in its intended weapon system to determine if it meets design specifications. 9. Actual system “flight proven” Actual application of the Actual system in final form Operational Test and Evaluation through successful mission technology in its final form and (OT&E) in operational mission operations under mission conditions, such conditions. as those encountered in operational test and evaluation. In almost all cases, this is the end of the last “bug fixing” aspects of true system development. Examples include using the system under operational mission conditions. Source: GAO and its analysis of National Aeronautics and Space Administration data.

Page 178 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix VI

Comments from the Department of Defense AppendixVI

Page 179 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix VI Comments from the Department of Defense

Page 180 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix VII

GAO Contact and Acknowledgments AppendixVII

GAO Contact Michael J. Sullivan, (202) 512-4841 or [email protected]

Acknowledgments Principal contributors to this report were Ronald E. Schwenn, Assistant Director; Raj C. Chitikila; Deanna R. Laufer; Alan D. Rozzi; and Wendy P. Smythe. Other key contributors included David B. Best, Maricela Cherveny, Bruce D. Fairbairn, Arthur Gallegos, William R. Graveline, Kristine R. Hassinger, Michael J. Hesse, Meredith A. Kimmett, C. James Madar, Stephen P. Marchesani, Jean L. McSween, Kenneth E. Patton, Charles W. Perdue, W. Kendal Roberts, Rae Ann H. Sapp, Roxanna T. Sun, Robert S. Swierczek, Bruce H. Thomas, and Karen S. Zuckerstein.

The following were responsible for individual programs:

System Primary staff Advanced Extremely High Frequency (AEHF) Satellite Bradley L. Terry AGM-88 Advanced Anti-Radiation Guided Missile (AARGM) Kathryn M. Edelman, Grant M. Sutton Air and Missile Defense Radar (AMDR) Molly W. Traci Apache Block III (AB3) Helena Brink Army Integrated Air and Missile Defense (Army IAMD) Carol T. Mebane, Ryan D. Stott B-2 Defensive Management System (DMS) Modernization Matthew P. Lea, Sean D. Merrill B-2 Extremely High Frequency (EHF) SATCOM Capability, Sean D. Merrill, Don M. Springman Increment 1 B-2 Extremely High Frequency (EHF) SATCOM Capability, Don M. Springman, Sean D. Merrill Increment 2 BMDS: Airborne Laser Test Bed (ALTB) LaTonya D. Miller BMDS: Flexible Target Family (FTF) Ivy G. Hubler, Teague A. Lyons BMDS: Ground-Based Midcourse Defense (GMD) Steven B. Stern, Rebecca Guerrero BMDS: Terminal High Altitude Area Defense (THAAD) Meredith A. Kimmett, Brian A. Tittle Broad Area Maritime Surveillance (BAMS) Unmanned Aircraft W. William Russell, Jodi G. Munson System (UAS) C-130 Avionics Modernization Program (C-130 AMP) Lauren M. Heft, Kathy Hubbell C-27J Joint Cargo Aircraft (JCA) Andrew H. Redd C-5 Reliability Enhancement and Reengining Program (C-5 RERP) Cheryl K. Andrew, Megan L. Hill CH-53K - Heavy Lift Replacement Marvin E. Bonner, Robert K. Miller CVN 21 Future Aircraft Carrier W. Kendal Roberts, Robert P. Bullock DDG 1000 Destroyer Deanna R. Laufer, W. Kendal Roberts DDG 51 Destroyer Molly W. Traci

Page 181 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix VII GAO Contact and Acknowledgments

(Continued From Previous Page) System Primary staff Defense Weather Satellite System (DWSS) Maricela Cherveny E-2D Advanced Hawkeye (E-2D AHE) Jeffrey L. Hartnett, Teague A. Lyons Enhanced Polar System (EPS) Bradley L. Terry Excalibur Precision Guided Extended Range Artillery Projectile Wendy P. Smythe Expeditionary Fighting Vehicle (EFV) Jerry W. Clark, MacKenzie H. Cooper F-22A Raptor Andrew H. Redd, Michael W. Aiken F-35 Lightning II (Joint Strike Fighter) Charlie Shivers, LeAnna Parkey Family of Advanced Beyond Line-of-Sight Terminals (FAB-T) Scott Purdy, Alexandra K. Dew Global Hawk (RQ-4A/B) Laura Jezewski, Travis J. Masters Global Positioning System (GPS) IIIA Laura T. Holliday, Laura Hook, Sigrid McGinty GPS III OCX Ground Control Segment Arturo Holguin, Jr. Gray Eagle Tana M. Davis H-1 Upgrades (UH-1Y/AH-1Z) Stephen V. Marchesani Increment 1 Early-Infantry Brigade Combat Team (E-IBCT) Marcus C. Ferguson, Tana M. Davis Intelligent Munitions System–Scorpion Wendy P. Smythe, John S. Warren Joint Air-to-Ground Missile (JAGM) Carrie W. Rogers Joint Air-to-Surface Standoff Missile (JASSM) John W. Crawford, Michael J. Hesse Joint High Speed Vessel (JHSV) J. Kristopher Keener, Erin E. Preston Joint Land Attack Cruise Missile Defense Elevated Netted Sensor John M. Ortiz System (JLENS) Joint Light Tactical Vehicle (JLTV) Danny G. Owens, Dayna L. Foster Joint Precision Approach and Landing System (JPALS) W. Kendal Roberts Airborne and Maritime/Fixed Station Joint Tactical Radio System Paul G. Williams (AMF JTRS) Joint Tactical Radio System (JTRS) Ground Mobile Radios (GMR) Nathan A. Tranquilli, Ridge C. Bowman Joint Tactical Radio System (JTRS) Handheld, Manpack, and Small Nathan A. Tranquilli, Ridge C. Bowman Form Fit (HMS) KC-X Program Wendell K. Hudson, Mary Jo Lewnard LHA Replacement Amphibious Assault Ship (LHA 6) Celina F. Davidson Littoral Combat Ship (LCS) John P. Dell’Osso, Christopher R. Durbin Littoral Combat Ship-Mission Modules Gwyneth B. Woolwine, Jeremy Hawk, Christopher R. Durbin Maritime Prepositioning Force (Future)/Mobile Landing Platform Erin E. Preston, J. Kristopher Keener Mobile User Objective System (MUOS) Richard Y. Horiuchi National Polar-orbiting Operational Environment Satellite System Suzanne Sterling (NPOESS) Navy Multiband Terminal (NMT) Lisa P. Gardner Navy Unmanned Combat Air System Aircraft Carrier Demonstration Julie C. Hadley, Travis J. Masters (UCAS-D) Nett Warrior Increment 1 William C. Allbritton

Page 182 GAO-11-233SP Assessments of Selected Weapon Programs

Appendix VII GAO Contact and Acknowledgments

(Continued From Previous Page) System Primary staff Ohio-Class Replacement (OR)/Sea Based Strategic Deterrent Alan D. Rozzi, C. James Madar P-8A Poseidon Heather L. Miller, Jacob L. Beier PATRIOT/Medium Extended Air Defense System (MEADS) Ryan D. Stott, Carol T. Mebane Combined Aggregate Program (CAP) Fire Unit Reaper Unmanned Aircraft System Rae Ann H. Sapp Ship to Shore Connector (SSC) Meghan Hardy, Kelly Bradley Small Diameter Bomb (SDB), Increment II Michael J. Hesse Space Based Infrared System (SBIRS) High Program Claire Buck Space Fence Peter E. Zwanzig Standard Missile-6 (SM-6) Extended Range Active Missile (ERAM) Angie Nichols-Friedman, Deanna R. Laufer Stryker Modernization (SMOD) Andrea M. Bivens, Wendy P. Smythe Three Dimensional Expeditionary Long Range Radar (3DELRR) Anne McDonough-Hughes, Amy Moran Lowe V-22 Joint Services Advanced Vertical Lift Aircraft (Osprey) Bonita P. Oden, Jerry W. Clark Presidential Helicopter (VXX) J. Andrew Walker, Michael W. Aiken Vertical Take-off and Landing Tactical Unmanned Aerial Vehicle Leigh Ann Nally (VTUAV) Virginia Class Submarine (SSN 774) C. James Madar, Alan D. Rozzi Warfighter Information Network-Tactical (WIN-T) Increment 2 James P. Tallon Warfighter Information Network-Tactical (WIN-T) Increment 3 James P. Tallon Source: GAO.

Page 183 GAO-11-233SP Assessments of Selected Weapon Programs

Related GAO Products

Defense Acquisitions: Observations on Weapon Program Performance and Acquisition Reforms. GAO-10-706T. Washington, D.C: May 19, 2010.

Defense Acquisitions: Strong Leadership Is Key to Planning and Executing Stable Weapon Programs. GAO-10-522. Washington, D.C.: May 6, 2010.

Best Practices: DOD Can Achieve Better Outcomes by Standardizing the Way Manufacturing Risks Are Managed. GAO-10-439. Washington, D.C: April 22, 2010.

Defense Acquisitions: Assessments of Selected Weapon Programs. GAO- 10-388SP. Washington, D.C.: March 30, 2010.

Defense Acquisitions: Many Analyses of Alternatives Have Not Provided a Robust Assessment of Weapon System Options. GAO-09-665. Washington, D.C.: September 24, 2009.

Best Practices: High Levels of Knowledge at Key Points Differentiate Commercial Shipbuilding from Navy Shipbuilding. GAO-09-322. Washington, D.C.: May 13, 2009.

Defense Acquisitions: Charting a Course for Lasting Reform. GAO-09- 663T. Washington, D.C.: April 30, 2009.

Defense Acquisitions: Measuring the Value of DOD’s Weapon Programs Requires Starting with Realistic Baselines. GAO-09-543T. Washington, D.C.: April 1, 2009.

Defense Acquisitions: DOD Must Balance Its Needs with Available Resources and Follow an Incremental Approach to Acquiring Weapon Systems. GAO-09-431T. Washington, D.C.: March 3, 2009.

GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs. GAO-09-3SP. Washington, D.C.: March 2, 2009.

Defense Acquisitions: Perspectives on Potential Changes to Department of Defense Acquisition Management Framework. GAO-09-295R. Washington, D.C.: February 27, 2009.

Page 184 GAO-11-233SP Assessments of Selected Weapon Programs

Related GAO Products

Defense Acquisitions: DOD’s Requirements Determination Process Has Not Been Effective in Prioritizing Joint Capabilities. GAO-08-1060. Washington, D.C.: September 25, 2008.

Defense Acquisitions: A Knowledge-Based Funding Approach Could Improve Major Weapon System Program Outcomes. GAO-08-619. Washington, D.C.: July 2, 2008.

Best Practices: Increased Focus on Requirements and Oversight Needed to Improve DOD’s Acquisition Environment and Weapon System Quality. GAO-08-294. Washington, D.C.: February 1, 2008.

Best Practices: An Integrated Portfolio Management Approach to Weapon System Investments Could Improve DOD’s Acquisition Outcomes. GAO- 07-388. Washington, D.C.: March 30, 2007.

Defense Acquisitions: Major Weapon Systems Continue to Experience Cost and Schedule Problems under DOD’s Revised Policy. GAO-06-368. Washington, D.C.: April 14, 2006.

Best Practices: Better Support of Weapon System Program Managers Needed to Improve Outcomes. GAO-06-110. Washington, D.C.: November 30, 2005.

Defense Acquisitions: Stronger Management Practices Are Needed to Improve DOD’s Software-Intensive Weapon Acquisitions. GAO-04-393. Washington, D.C.: March 1, 2004.

Best Practices: Setting Requirements Differently Could Reduce Weapon Systems’ Total Ownership Costs. GAO-03-57. Washington, D.C.: February 11, 2003.

Best Practices: Capturing Design and Manufacturing Knowledge Early Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.: July 15, 2002.

Best Practices: Better Matching of Needs and Resources Will Lead to Better Weapon System Outcomes. GAO-01-288. Washington, D.C.: March 8, 2001.

Page 185 GAO-11-233SP Assessments of Selected Weapon Programs

Related GAO Products

Best Practices: A More Constructive Test Approach Is Key to Better Weapon System Outcomes. GAO/NSIAD-00-199. Washington, D.C.: July 31, 2000.

Best Practices: Better Management of Technology Development Can Improve Weapon System Outcomes. GAO/NSIAD-99-162. Washington, D.C.: July 30, 1999.

Best Practices: Successful Application to Weapon Acquisition Requires Changes in DOD’s Environment. GAO/NSIAD-98-56. Washington, D.C.: February 24, 1998.

Best Practices: Commercial Quality Assurance Practices Offer Improvements for DOD. GAO/NSIAD-96-162. Washington, D.C.: August 26, 1996.

(120920) Page 186 GAO-11-233SP Assessments of Selected Weapon Programs

GAO’s Mission The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability.

Obtaining Copies of The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site (www.gao.gov). Each weekday afternoon, GAO GAO Reports and posts on its Web site newly released reports, testimony, and correspondence. To have GAO e mail you a list of newly posted products, Testimony go to www.gao.gov and select "E-mail Updates."

Order by Phone The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's Web site, http://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information.

To Report Fraud, Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm Waste, and Abuse in E-mail: [email protected] Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470

Congressional Ralph Dawn, Managing Director, [email protected], (202) 512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125 Relations Washington, DC 20548

Public Affairs Chuck Young, Managing Director, [email protected], (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548