PCEC V2.3 2019 NASA Cost & Schedule Symposium &Schedule Costnasa 2019 Brian Alford Brian Shawn Hayes Shawn TGS Consultants TGS Booz Allen Allen Hamilton Booz Overview

C SF M SA

i Team t

S SBDiversity

MIPSS

Mark Jacobs Mark t c i V Richard Webb Richard KAR KAR Enterprises TGS Consultants TGS August 15, 2019 15, August PCEC PCEC v2.3 2019 NASA Cost & Schedule Symposium &Schedule CostNASA 2019 Brian Alford Brian Shawn Hayes Shawn TGS Consultants TGS Booz Allen Allen Hamilton Booz Overview

• PCEC Overview • Robotic Spacecraft Updates • Crewed & Space Transportation System Update • Summary

Victory Solutions MIPSS Team 2 What is PCEC?

• The Project Cost Estimating Capability (PCEC) is the primary NASA-sponsored parametric cost tool for spacecraft estimates – Developed and maintained by NASA at MSFC beginning in late 2013 – Excel Add-in that provides capabilities and cost estimating artifacts used to build a spacecraft cost estimate in Excel – Based on more than 70 missions/system elements, but with separate approaches for modeling different types of systems • Robotic Spacecraft (Robotic SC) • Crewed & Space Transportation Systems (CASTS) – Completely transparent tool: no code passwords, protected sheets, etc. – Available to the general public via ONCE and the NASA Software Catalog (https://software.nasa.gov/)

Victory Solutions MIPSS Team 3 Current State of PCEC

• PCEC v2.2.1 is the current version, released in May 2018 – Continues to be distributed, supported, and used on of a variety of analyses, AOs/proposals, design competitions, etc. • Current user counts as of late July 2019 – 575+ Users / Requestors – 45+ Countries represented – ~85 users added since last year’s Symposium • PCEC v2.3 is in currently in development; release currently planned for this Fall

Victory Solutions MIPSS Team 4 PCEC User Distribution As of Jul 2019

Estimated based on user-provided data

Victory Solutions MIPSS Team 5 PCEC v2.3 Overview of Planned Updates

• Enhancements/Updates to Functional Capabilities – New Robotic SC Normalizations – Updates to Robotic SC CERs – Support for linking to latest versions of NICM, MOCET – Estimating Template Reorganization – Updates to WBSs, Globals, & Subsystem Inputs worksheets • New Capabilities – Entry System CERs – Linking to Propulsion Cost Model (Liquid Rocket Engine version) – Library Loading/Unloading – Optional S-curve Template – Bug Fixes

Please give us feedback on this list!

Victory Solutions MIPSS Team 6 PCEC ROBOTIC SPACECRAFT UPDATES

7 PCEC for Robotic Missions Data Collection/Normalization Refinements

• Changes were needed in the normalization workbooks and subsystem-level inputs used for the PCEC CERs

• Workbook changes: – Removed all RTG costs (Cassini, NH, MSL) – Isolating costs for items captured outside the s/s CERs (TPS, Parachutes, Air Bags, EP Thrusters/PPUs, SRMs – affects Phoenix, MER, MSL, Dawn, NH) and removed these from the s/c-level MEL- based heritage assignments

• Input changes were made to ensure all costs and technical performance metrics are accurate and representative of what is intended to be captured – All Inputs & Costs used for CER development have been captured/documented in a single excel file database

Victory Solutions MIPSS Team PCEC for Robotic Missions Missions in Current Database

• Current set = 49 missions/58 flight systems • Limited data for several missions – Subsystem breakouts not available for JPSS-1, TESS, Solar Probe, and ICON (all are new mission candidates) • Several missions with multiple flight elements did not breakout $s for each – Includes InSight, OSIRIS-REx, CYGNSS – Supporting data was used to allocate CADRe data to individual flight elements • Updates include data refinements to ensure consistency and improve accuracy

Key: Victory Solutions MIPSS Team PCEC for Robotic Missions Data Collection/Normalization Challenges/Issues

• CADRe data for several missions does not include sufficient detail to derive accurate subsystem allocations – Subsystem data not included/updated – Costs for multiple flight elements are often combined – S/C contract tracked as a single WBS line (only JPSS-1) • Breakouts of Cruise Stage and Entry System costs from LM for InSight would be very helpful – InSight is a unique data point in that it attempted a nearly full Build-to-Print – Phoenix was different since it leveraged a significant amount of residual flight hardware from MSP’01 – InSight cost details could also be used to properly assess the value of the residual hardware used on Phoenix • CADRe needs more emphasis on identifying subsystem costs by major flight element on current missions – Consider an external effort (or site visits) to collect missing data for past missions? – Make this an AO requirement for future missions?

Victory Solutions MIPSS Team PCEC for Robotic Missions Secondary Flight Systems

Goal is to better estimate pieces of complex robotic systems

MSL Deep 1) Cruise Impact Stage 1) Carrier 2) Backshell 2) 3) EDL Impactor 4) Rover 5) TPS 6) Parachute

• CERs specific to Secondary Element subsystems were explored, but confidence was low given the small data set

Victory Solutions MIPSS Team PCEC for Robotic Missions CER Improvements for Secondary Flight Systems

Secondary Flight • Feedback from multiple users have noted PCEC Elements seems to overestimate secondary flight systems – Includes: Cruise Stages, EDL, S/C Carriers – 9 items identified in PCEC data – Most significant impacts appear to be for ‘Comm’ and ‘GNC’ • PCEC performance has been tested against these actuals and seems biased high, even with use of the TPS/Parachute CERs, which significantly improve PCEC performance for ‘Str&Mech’ Secondary Flight Elements Subsystem • Identification of secondary element subsystems has Definitions been added as an input candidate for the updated CERs – Updates include the revised normalization data and other input candidates – Preliminary results show significant potential improvements

Victory Solutions MIPSS Team PCEC for Robotic Missions Approach for CER Updates

• Updated several data normalization incorporating feedback and new info – Utilized data in the CADRe supporting files to assist allocations to flight elements and subsystems – Identified unique component types that are not handled well by the PCEC CERs – Unique component types: RTGs, Ion Engines/PPUs, TPS, Parachutes, and Solid Rocket Motors – These items are typically estimated separately (PCEC CERs have been developed for TPS and Parachutes) • Reevaluate CERs using the 7 new data points including an additional input for Secondary Flight System – Initially tried to just add a Secondary Flight System input (Y/N) to the current CER input sets, but results did not show an improvement over current PCEC CER version – Reassessed CERs (using PCA and other approaches) using a broader set of initial input candidates – Compared approaches with NRC & RC separated to a single subsystem CER covering both • Comparing performance of new CERs to PCEC v2.2.1 CERs – Performance characterization and comparison efforts are in progress Victory Solutions MIPSS Team PCEC for Robotic Missions CER Development – Current Status

• It has been difficult to improve performance of the current PCEC CERs – A significant portion of the secondary flight system overestimates appears to be resolved if the TPS/Parachute CERs are used (these will be integrated with the next PCEC update) – New data points appear to be reasonably captured by PCEC v2.2.1 – Updated CERs incorporating the new data points (includes 49 missions and 58 flight systems) show minor improvements

• Although statistical performance seems good, differences compared to actuals has not improved – Updated CERs appear to have a high-estimate bias Victory Solutions MIPSS Team PCEC for Robotic Missions CER Development – Options

• Two options are being explored – Develop CERs for Total Subsystem cost and then derive spreading functions – Use $/kg as the dependent variable • The option to use $/kg as the dependent variable seems promising – Allows greater sensitivity to other inputs and reduces input inter-dependencies – This effort has only been explored for the Total Cost (not NRC & RC independently) • Preliminary comparisons of estimating error do not show an improvement over the current PCEC CERs yet – Modeling Total Cost shows only minor improvements over NRC+RC – Using $/kg as the dependent variable looks better but needs more development – Next step will be to explore NRC & RC CERs for $/kg

Victory Solutions MIPSS Team PCEC for Robotic Missions CER Development Challenges/Issues

• Selection of Inputs – PCA and other approaches are helpful for identifying inputs, but expert judgement is needed to assess whether the inputs are sensible and ‘causal’ (not ‘associative’) – Difficult to address non-linearity • Handling of Categorical Inputs – Software that handles categories often has their influence backwards due to inter-relationships among the inputs – Assigning rankings to these categories is very qualitative and relative rankings can have significant impacts • Data Inconsistencies can significantly affect CER accuracy – Although the data has been normalized, each mission has several unique characteristics that are difficult to accurately capture – Data for older missions may be incomplete – Data for newer missions does not always breakout costs for individual flight system elements Victory Solutions MIPSS Team PCEC for Robotic Missions Future Plans

• Complete updates to the S/C subsystem CERs

• PM/SE/MA/I&T CER Updates – Including the 7 completed normalization data from recent missions

• Collect & Normalize data for New Missions – TESS, Solar Probe, ICON

• Quality Improvement for Select Missions – Attempt to collect missing cost details

Victory Solutions MIPSS Team PCEC CASTS MODEL UPDATES

18 Summary

• Propulsion Cost Model – Overview and Status

• Historical Manned Systems Documentation – CADRe-like documentation of historical manned transportation systems – Initial systems • Shuttle Orbiter • Lunar Command Service Module • Lunar Module – Status • Prototype Parts A, B, and C developed • Shuttle Orbiter completed - draft

Victory Solutions MIPSS Team 19 Propulsion Cost Model (PCM) Overview

• What is PCM? – Add-in to CASTS • Standalone model to PCEC/CASTS • Linkable to PCEC estimate similar to NICM, MOCET • PCM Capability – Liquid Rocket Engines, Solid Rocket Motors, Nuclear Thermal Propulsion, . . . – Non Recurring DDTE + First Unit + Production $’s • PCM Availability – Similar approach to PCEC/CASTS – Publicly released model (spreadsheet) – Unrestricted and Restricted documentation • Manual plus (restricted) source database/calibrations

Victory Solutions MIPSS Team 20 PCM Status

• Liquid Engines – Completed – final model testing within MSFC ECO – Calibrated to historical database – Documentation undergoing final edits • Nuclear Thermal – CERs completed – initial prototype model undergoing testing – Documentation in-work • Solid Motors – In-work CERs • Add small solids to CASTS database Complete

Propulsion Cost Model

Liquid Solid Nuclear Engines Motors Themal Other? Engine Cycle Monolithic Thermionic Hypersonic Thrust Segmented Thermoelec RBCC Propellants Small Cycle Ion Test Approach Total Impulse Thermal Ctl Solar Sail Victory Solutions MIPSS Team 21 PCM Liquid Engines Summary

• Based on Liquid Rocket Engine Cost Model (LRECM) – Developed by Rocketdyne (circa 1992-2003) – “Bought” by NASA mid-90’s; updated mid ‘00’s – “Engineering” model – limited number data points • Modifications for PCM version – Adding additional data points + Modifying/changing CER’s • Propellant combinations, pressure (versus pump) – fed Reus Launch Engine Cycle Propellants /Expend System(s) Average Unit Cost Calibrations F1 Gas Generator RP/LOX Expend Saturn V MA5 Gas Generator RP/LOX Expend Atlas II RS27 Gas Generator RP/LOX Expend Delta II J2 Gas Generator LH2/LOX Expend Saturn II J2X Gas Generator LH2/LOX Expend not apply RL10A3 Split Expander* LH2/LOX Expend Multiple RS68 Gas Generator LH2/LOX Expend Delta IV LR87 Gas Generator Hypergolic Expend Titan IV LR91 Gas Generator Hypergolic Expend Titan IV Viking VI Gas Generator Hypergolic Expend Ariane 4, 5 SSME Stg Combustion (2 shaft) LH2/LOX Reus Shuttle RD180 Stg Combustion (1 shaft) RP/LOX Expend Atlas V LM Ascent Pressure-Fed Hypergolic Expend Lunar Module LM Descent Pressure-Fed Hypergolic Expend Lunar Module OMS Pressure-Fed Hypergolic Reus Shuttle RL10C1 Split Expander* LH2/LOX Expend Multiple Victory Solutions MIPSS Team 22 PCM Liquid Engines Documentation

• Manuals – Restricted and Unrestricted versions (similar to CASTS) • Restricted = Unrestricted + historical cost database and variable inputs • Technical Data Sheets for each engine in historical database • Unrestricted – publicly available data • Summary Overview & History • Engine Overview, Development, Schedules, Key Milestones, etc. • System Parameters • Cycle, Dimensions, Weight, Thrust, ISP, Fuel/Oxidizer, Mixture Ratio, etc. • Production History • Number produced/flown, production rates per year, etc. • System Description • Specific technical information

Victory Solutions MIPSS Team 23 PCM Nuclear Thermal Summary

• Based on “Cost Estimating Relationships for Nuclear Power and Propulsion” report (1992, REDSTAR 0131-00602) – Developed by Rocketdyne under contract to Lewis Research Center – “Engineering” model – limited number data points – ROVER/NERVA, SNAP, SP-100 – Includes both fixed plant/in-situ and transportation system CERs • PCM includes only transportation system (currently)

Example Output Primary Cost Contributor: Thruster M 15$'s Dev Unit TOTAL Reactor, Safety, Chamber, Nozzle, Nozzle Extension, Propellant Supply$ 435 $ 355 $ 790 Integ & Test Thruster $ 2,029 $ 360 $ 2,389 NTR Thruster DDTE Structures $ 116 $ 16 $ 132 $2,250 Control/Condition Monitor $ 68 $ 24 $ 92 $2,200 SUBTOTAL $ 2,649 $ 755 $ 3,403 $2,150 Ground Test Hardware$ 830 $ 830 $2,100 Ground Test $ 696 $ 696 $2,050 SEI $ 835 $ 835 $2,000

Assembly $ 75 $ 75 DDTE (M15$'s) $1,950 Acceptance Test $ 125 $ 125 $1,900 PM&S$ 48 $ 48 $1,850 TOTAL $ 5,009 $ 1,002 $ 6,012 - 1,000 2,000 3,000 4,000 5,000 6,000 Power Output (MWth) Victory Solutions MIPSS Team 24 Historical Manned Systems Documentation - Summary

• CADRe-like documentation of historical Candidate Systems / exploration systems Missions Apollo Command & – Initiated by/coordinated with Agency Service Module Programmatic Analysis Research and Capability Apollo Lunar Module (APARC) office Shuttle Orbiter – Includes parts A, B, and C similar to CADRe but Skylab tailored to historical exploration systems Spacelab – Based on data in REDSTAR and public sources External Tank – To be made available to NASA cost community Centaur G’ through One NASA Cost Engineering (ONCE) Titan Centaur and the REDSTAR Library S-IC • Status S-II S-IVB – Prototype Parts A, B, and C templates Centaur D developed Engines – Shuttle Orbiter completed - draft

Victory Solutions MIPSS Team 25 Historical Manned Systems Documentation – Part Detail

Part A (Word/PDF) Part B (Spreadsheet) Part C (Spreadsheet) Overview & History System Level • Life Cycle Cost in • Program/System development and Parameters Project WBS production historical overview • Schedule • Life Cycle Cost in • Description of key project • System Hardware Historical Manned characteristics, events, unique WBS Systems Standard circumstances • Software WBS • E.g. Test Program • WBS Dictionary • Yearly Funding • Development and Production • Manpower Profiles schedules and key milestones • Primary Cost System Summary REDSTAR Reference Assumptions (source • NASA management, prime Documents data) contractor, contract arrangements, • Schedule key suppliers, production rates, etc. • REDSTAR Reference Key Parameters Summary Documents • Length/Width/Height, system-level • All Costs in Real Year mass, performance characteristics, $’s mission profile Subsystem Descriptions • Detailed subsystem descriptions

Victory Solutions MIPSS Team 26 Historical Manned Systems Documentation – Example

• Example: Shuttle Orbiter Parts A, B, C (draft)

Manned Space Transportation Systems Historical Data Manned Space Transportation Systems Historical Data Shuttle Orbiter Part B Shuttle Orbiter Part C Exploration Data Fields RY $'s Non Recurring Recurring Total 1.3.1 Value STRUCTURE $ 1,029 $ 435 $ 1,464 System 1.3.1.1 PRIM STRUCTURE$ 614 $ 257 $ 871 Name Space Shuttle Orbiter .1 FWD FUSE $ 34 $ 28 $ 62 Human Rated Yes .2 MID FUSE $ 174 $ 48 $ 222 .3 AFT FUSE$ 121 $ 84 $ 205 LEO from 100nm to 280nm; azimuth.4 CM up to 62 degrees $ 75 $ 46 $ 121 Destination (Space Station = 51.6) .5 WINGS$ 150 $ 33 $ 183 Recovery Mode glideback; horizontal landing .6 TAIL $ 60 $ 18 $ 78 Re-usable or Expendable? Reusable 1.3.1.2 TPS$ 415 $ 179 $ 594 .1 EXT INS $ 312 $ 141 $ 453 Mass Dry approx 150,000lb; varies by Orbiter .2 L EDGE$ 74 $ 25 $ 99 Mass Wet .3 INT INS $ 29 $ 13 $ 42 Major Subsystem Quantity 1 1.3.2 PROPULSION$ 512 $ 212 $ 723 Payload Capability up = 53,700 to LEO; down = 31,700.1 MAIN ENG $ 116 $ 44 $ 159 Number of Government Organizations NASA; JSC lead development .2center RCS$ 138 $ 69 $ 207 .3 OMS $ 256 $ 99 $ 355 Rockwell International (RI) Prime Contractor(s) .4 ABE$ 2 $ 0 $ 2 New Design Rating New 1.3.3 POWER $ 174 $ 51 $ 225 .1 PWR GEN$ 80 $ 31 $ 111 Launch Abort System 4 abort modes (RTLS, TAL, AOA,.2 APU ATO - see Part A) $ 94 $ 20 $ 114 Downey, CA (RI manufacturing),1.3.4 AVIONICS Palmdale, CA$ (final 1,397 $ 343 $ 1,740 .1 GN&C $ 438 $ 76 $ 514 Production Facility Locations assembly) .2 C&T $ 328 $ 71 $ 399 KSC for STS vehicle flight-to-flight;.3 D&C Palmdale, CA for $ 144 $ 51 $ 195 Integration Locations Orbiter final assembly .4 INSTRU $ 87 $ 24 $ 111 Launch Location KSC LC39A, B .5.1 DMS$ 109 $ 35 $ 143 OMS Monomethyl Hydrazine (MMH);.5.2 S/W Orbiter $ 104 $ - $ 104 .6 EPD&C$ 188 $ 87 $ 274 Fuel Type (applicable to boosters) MPS+SSME LH2 1.3.5 ECLSS $ 153 $ 70 $ 222 OMS Nitrogen Tetroxide (N2O4);.1 ATMOS Orbiter MPS+SSME $ 69 $ 32 $ 102 Oxidizer Type LOX .2 LIFE SPT $ 15 $ 7 $ 22 Engine(s) Yes .3 THERMAL$ 67 $ 28 $ 95 Orbit Maneuvering System (OMS);.4 AIRLOCK Space Shuttle Main $ 2 $ 3 $ 4 1.3.6 CREW STA$ 38 $ 38 $ 7 7 $ 45 $ 45 Engine Name Engines (SSME) 1.3.7 MECH SYS $ 293 $ 87 $ 379 Developer OMS: Aerojet; SSME: Rockwell.1 LRocketdyne GEAR$ 27 $ 11 $ 38 New Design Rating OMS: heritage Apollo CSM; SSME:.3 ACT SYS new $ 76 $ 29 $ 104 .5 HYDR SYS$ 93 $ 26 $ 119 .7 P/L BAY DOOR $ 50 $ 9 $ 59 Victory Solutions MIPSS Team .8 SEP SYS$ 47 $ 12 27 $ 59 SUBTOTAL HARDWARE $ 3,594 $ 3,594 $ 1,205 $ 1,205 $ 4,798 $ 4,798 CLOSING

28 Some Key Questions / Issues to Work Through

• Since CERs are changing, how will this impact existing estimates? – We want to be explicit about communicating what changed and why • How do we (should we?) continue to provide access to old CERs in the latest version of the tool? – Include them alongside the new CERs? – Make the Library swappable by the user? – Require users to use previous versions of the tool? • How do we support users who have existing estimates that were developed in v2.2.1 & prior but may to be updated using v2.3? – Need to work through use cases of estimates that might span multiple versions and potential issues that might arise – Might involve routines to “convert” old estimates into new estimates? Estimates will change if opening them with the latest CERs even if no program changes

Victory Solutions MIPSS Team 29 Next Steps & Summary

• PCEC v2.3 will be completed by the end of the year with key updates to Robotic SC estimating, linkages to new versions of external models, and improvements in usability

• Activities to complete PCEC v2.3 – Finalize the Robotic SC CER updates & incorporate into Library – Complete development and testing of proposed updates & enhancements – Update documentation (Help files, User guides, CER workbooks) • Other Near Term Steps – Continue work on other PCM modules (NTP & Solids) – Support PCEC Training Course Development – Complete historical documentation and provide for ONCE/REDSTAR

Victory Solutions MIPSS Team 30 Closing

• Please provide us feedback! – Not too late to help v2.3 release: development ideas, testing, etc.

• Questions?

• Demo: TODAY at 10:30 in Gemini 1

PCEC Email Contact: MSFC-PCEC@mail.nasa.gov Application Website(s): ONCE (NASA Civil Servants) https://software.nasa.gov/ , search for PCEC

Victory Solutions MIPSS Team 31