Aligning Perspectives and Methods for Value-Driven Design

Adam M. Ross, M. Gregory O’Neill, Daniel E. Hastings, and Donna H. Rhodes

September 1, 2010

Track 100 -MIL-4: Emerging Trends AIAA Space 2010 Outline

• Introduction • Defining Value • Economic Background • Overview of VCDMs • Spacecraft Mission Examples • Discussion

• What is the most valuable system for our mission? – Value is perceived by stakeholder(s) of interest – Value can be comprised of both direct and indirect benefits and costs • Formulating a response: “valuation” methods – “Traditional” requirement- and cost-centric approaches – Value-driven design approaches • Theory: economics, decision analysis, psychology, behavioral economics, and complex engineering design • Operationalization: value-centric design methodologies (VCDMs)

Objective Domain Engineering Design Domain Stakeholder Most Valuable Candidate “Valuation” Value System Design System Designs Method

• Relevancy: notable efforts in the aerospace sector – DARPA System F6 Program and valuation of fractionated spacecraft – Four industry-lead teams developed and applied VCDMs seari.mit.edu © 2010 Massachusetts Institute of Technology 3 Defining Value

• “Value” definition includes several important factors – Cost/Resources HlitidfiitiHolistic definition of f“ “creati ng val ue” – Satisfaction/Utility Balancing and increasing the net level of – Importance/Priority (1) satisfaction, with (2) available resources, while addressing (3) its degree of importance • Value in design is subjective and context-dependent – Example: “value” of heated seats in Maine winter vs. Florida summer

• Evolution of definition in economic theory – Value in use vs. value in exchange – Scarcity vs. labor requirement – Willingness to pay (i.e., “pr ice ” vs. wea lth) – Supply-side vs. demand-side – Marginal utility vs. marginal cost

S= Supply Many demanders Single demander D= Demand

OfferingsS D Preferences OfferingsS D Preferences liers pp any sup MM Exchange Value creation in design

OfferingsS D Preferences OfferingsS D Preferences plier

? ingle sup SS • In order for suppliers and demanders to experience “value” creation in system design, exchanges must occur •Whyyppppy does supplier participate? Why does demander? • The smaller the market, the more the suppliers and demanders must understand “specifics” in order for “enough” exchanges to occur seari.mit.edu © 2010 Massachusetts Institute of Technology 5 Operationalizing in the Aerospace Context

• Several market types present in aerospace domain context – Commercial (e. g., communications) often more competitive market with many suppliers and many demanders (e.g., consumers) – Defense/Intel (e.g., aircraft, satellites) often more restricted markets with few suppliers and few (or single) demanders (e.g., governments) – SiScience (bti)fttitdktithil(e.g., observatories) are often restricted markets with single suppliers and single demanders • Depending on the “mission,” the type of market will vary • The market type can help guide an analyst on an appropriate operationalization of “value” – if “price” is well-defined (such as in commercial markets), then it can be used as a proxy for value in exchange – If “price” is not well-defined or inappropriate (such as in science markets), then an alternative to dollarization may be necessary

Deppgending on the mission , different market t ypyppy;ypes may apply; the market type indicates appropriate metrics for value seari.mit.edu © 2010 Massachusetts Institute of Technology 6 Overview of VCDMs (1) Various Value-Centric Design Methodologies (VCDMs) can be found in use for operationalizing value for design • Net Present Value

tk tk tk Dt Dt Dt NPV  D  dt ~ D  ~ D  o t o t o t t j 1  r t 1  r t 1  r t j t j – “Value”: discounted cash flow – Limitations: revenue-centric, forecasting, uncertainty, aggregation,… • MltiMulti-Attr ibu te Utility Theory (w ith cos t)

 n n K U X 1   K  ki U X i 1  whe re, K  1  K  ki 1  i1 i1

– “Value”: aggregation of (non-monetary) benefits relative to monetary costs of achieving those benefits – Limitations: ordinality, aggregation, abstraction, independence,…

• Cost-Benefit Analysis

 tk Bt tk Ct     CB   Bo  Co   t   t  1 r 1 r   t j t j  – “Va lue ”: discount ed monet ary diff erence be tween se t o f mone tize d benefits and their monetized costs – Limitations: monetization, forecasting, uncertainty, distribution, … • Other (fdtil)(see paper for details) – Cumulative Prospect Theory (CPT) – Value functions (VFs) – Analytical Hierarchy Process (AHP) – Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) This list is not complete; a multitude of valuation methods exist (e.g., from finance, economics, marketing, decision analysis, etc.) seari.mit.edu © 2010 Massachusetts Institute of Technology 8 Comparing Common VCDMs

Table can be used as guide to help choose appropriate VCDM

All VCDMs have limitations and inherent assumptions. Goal should be thoughtful selection of most appropriate method. seari.mit.edu © 2010 Massachusetts Institute of Technology 9 Spacecraft Mission Examples

Telecommunications Telecommunications Mission Range (least to Attribute Definition Units most desirable) Stakeholder: Satellite owner and Operational duration of 1 Mission Lifetime years [5, 15] spacecraft Max Number of High Max number of distinct HD service provider 2 - [0, 100] Definition Channels channels available for broadcast Max Number of Standard Max number of distinct SD (e.g., DirecTV®, Inc.) 3 - [0, 200] Definition Channels channels available for broadcast Max Downlink Internet Max downlink internet 4 Mbps [0, 1080] Photo: Boeinggp 702HP, http://www.boein g.com/defense- Bandwidth bandwidth (aggregate) Uplink Internet Max uplink internet bandwidth space/space/bss/factsheets/702/ 5 Mbps [0, 1080] dtv10_11_12/dtv10_11_12.html Bandwidth (aggregate) Value proposition: provide high quality broadband entertainment to subscribers in North America for at least five years using one satellite, while maximizing profit

Deep space Deep-Space Observation Mission Range (least to Attribute Definition Units most desirable) Stakeholder: Civil science agency Operational duration of 1 Mission Lifetime years [5, 15] spacecraft Level of pointing accuracy about ((ge.g., NASA ) 2 Payload Pointing Stability rd [1e011e06][1e-01, 1e-06] dominant spacecraft inertia axis Min. angular separation of two 3 Angular Resolution mas [1000, 1] points that can be differentiated Photo: Hubble ST, Rate of rotation about dominant 4 Slew Rate rd/s [2e-06, 1e-01] http://www.nasa.gov/worldbook/hubble_ spacecraft inertia axis telescope_worldbook.html Aperture focal length relative to 5 Focal Ratio - [f/40, f/1]* its major axis diameter

Value proposition: provide visible wavelength images of astrophysical (stellar) phenomenon, in deep space (i.e., extragalactic) for support of scientific studies for at least five years

Launch Vehicle Input Group Model Algorithms Model Algorithm 123456789101112131415 Inputs 1 SET Inputs Physics-based Model Cost Model 2 Mission Payload(s) X Spacecraft 3 Computer System, C&DH X X Architecture Launch Vehicle 4 CitiTT&CCommunications, TT&C X X Feedback Payload Subsystems 5 ADS, GNS X Physics- Loops Input Group 6 EPS X X (Bus) Based COCOMO II 7 Propulsion, ACS, GCS X X XX Mass Model Power 8 TCS XXXXXXX X Parametric CER’s 9 Power Required XXXXXXXX XX Lifecycle & 10 Mass XXXXXXXX Size/Volume 11 Size, Volume XXXXXXXX X Design Input 12 LV Selection XXXXXXXX XX Cost 13 COCOMO II X X Group Model 14 Parametric CERs XXXXXXXX X Outputs 15 SET Outputs XXXXXXXXXXXXX Component Subsystem Module System Level Outputs Level Outputs Level Outputs Level Outputs • Various spacecraft designs proposed and assessed using Spacecraft Evaluation Tool (SET)* – High fidelity spacecraft modeling tool developed to simulate performance of monolithic or fractionated spacecraft architectures – 315 designs for telecom mission; 192 for deep space mission • Assessed metrics include – Lifecycle cost (both NRE and RE) – Attributes ((gboth emergent and controlled )

*O'Neill, M.G., "Spacecraft Evaluation Tool Verification and Validation," MIT SEAri WP-2010-3-2 (http://seari.mit.edu), Cambridge, MA, January 2010, seari.mit.edu © 2010 Massachusetts Institute of Technology 11 Results: Telecom Net Present Value/Cost-Benefit Analysis 13 1150 Most 11 1050 950 9 Value

8$M) 850

00 7 750 5 650 3 550 1 450 -1 350 250 -3 CB/NPV (FY2008$M) 150 sh Flow or DCF (FY20 DCF or Flow sh -5 aa 50 C -7 Cash Flow -50 -9 Discounted Cash Flow -150 -11 -250 01234567891011 1 23 45 67 89 111 133 155 177 199 221 243 265 287 309 Time (years) Spacecraft Design Tag (-) Multi pl e Att rib ut e Utilit y Theory Insights

0.70 – MAUT ignores revenue streams, 0.65 Candidate Designs but accounts for cost 0.60 Pareto Set 0.55 – NPV/CBA biased to attributes (-) yy 0500.50 Most 0.45 responsible for mos t revenue 0.40 Value – MAUT (leads to trades) vs. 0.35 0.30 NPV/CBA (selected design) 0.25 Multi-Attribute Utilit Multi-Attribute 0200.20 – Demonstrates the potential for 0.15 competing valuation structures via 0.10 275 325 375 425 475 525 575 differing VCDMs Deterministc Lifecycle Cost (FY2008$M) seari.mit.edu © 2010 Massachusetts Institute of Technology 12 Recommendation: Telecom

0.70 •MAUT 0.65 Candidate Designs – Supply only SD 0.60 NPV/CBAMostNPV/CBA Most 'Valuable' channels (max benefit 0.55 per transponder) MAUT Most 'Valuable' 0.50 • NPV/CBA 0.45 0.40 – Supply only HD ute Utility (-) ute Utility 0.35 channels (max revenue 0.30 per transponder) 0.25 • Mutually exclusive

Multi-Attrib 0.20 recommendations 0150.15 – Caveat: dependent on 0.10 assumed stakeholder 275 325 375 425 475 525 575 preference structure and Deterministc Lifecycle Cost (FY2008$M) market demand • CidtifRConsiderations for Recommen dtidations – Inherent assumptions in each VCDM – Variance in value quantification amongst and within VCDMs – First-order: differing cardinal/ordinal measures – Second-order: implicit uncertainty in “value” due to assumptions

Net Present Value/Cost-Benefit Analysis • Insights Net Present Value – Literature has not demonstrated 1. Quantify the initial investment for the spacecraft D o 2. Quantif yy() the discount rate (inflation and real ) r how to monetize attributes 3. Quantify revenue (profit minus cost) generated over the course of lifecycle as a function of time D t  – MAUT is readily executable 4. Compute the NPV via Eq. (1) • Recommendations Cost-Benefit Analysis 1. Quantify the discount rate (inflation and real) r – MAUT: long-lived system with 2. Quantify the cost of the spacecraft in time C t  reasonable, bu t no t the bes t 3. Repeat for all attributes attribute values to keep cost down - Quantify monetary value of attribute i in time vxi , t  B t  4. Compute the CB via Eq. (3) – NPV/CBA (no recommendation) Multi pl e Att rib ut e Utilit y Theory Mu ltipl e Att rib ut e Utilit y Theory (cos t inc l. )

0.75 0.75 0.70 Most 0.65 0.65 Value

Most (-) 0.60 (-) yy yy 0.55 Value 0.55 0.50 0.45 0.45 0.40

0.35 Candidate Designs Utilit Multi-Attribute 0.35 ulti-Attribute Utilit

MM Pareto Set 0.30 0.25 0.25 200 225 250 275 300 325 350 375 400 425 450 475 1 16 31 46 61 76 91 106 121 136 151 166 181 Deterministc Lifecycle Cost (FY2008$M) Spacecraft Design Tag (-) seari.mit.edu © 2010 Massachusetts Institute of Technology 14 Discussion

• Pick appropriate method given perspective – Choose meaning for “value” – Identify market “type” – Compare assumptions; weigh practicality with limitations • Insights from case studies may be representative – Possible spectrum based on market “type” from commercial to military to science missions • VDD Example: DARPA System F6 Program – Goal: develop and apply VCDM tools to provide risk-adjusted, net value comparison of monolithic vs . fractionated spacecraft – Four teams participated in Phase I, each with own VCDM – Each team used different valuation approach, implicit assumptions on “value” definition – Each came to different conclusions on the “value” of Source: fractionated spacecraft; likely due to disparities in value http://www.darpa.mil/tto/progra perspectives and methods ms/systemf6/index.html

Differences in value -driven design recommendations can arise not only from technical issues (e.g., data and modeling), but also from valuation method employed

• No single VCDM can provide “perfect” quantification of “value” – Each has assumptions and limitations – Each may apply to different interpretations on “value” definition – May apply differently depending on market type • Important to explicitly define “value” as well as reveal assumptions in method to quantify • May be useful to leverage multiple VCDMs in order to gain insight across limitinggp assumptions as well as to hel p communicate to different communities • It is essential to be consistent in application and communication of methods in order to properly compare results across studies and increase confidence in results

Balancinggp practicality , rig or, and app rop riateness, alig ning gp pers pectives and methods should be an explicit activity for value-driven design