Outline Introduction Printed Circuit Boards and Classification

National Aeronautics and Space Administration Outline

Printed Wiring Board Technology Infusion • Printed circuit board technology and Supplier Capability Overview • PCB quality assurance • Supplier capability study Bhanu Sood Commodity Risk Assessment Engineer • New technology insertion/TRL Risk and Reliability Branch • Risk based methods Quality and Reliability Division NASA Goddard Space Flight Center • Closure NASA Safety Center Webinar June 12, 2018

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300

Safely Achieve Amazing Science www.nasa.gov Distributionstatement:Approvedforpublicrelease. Through Mission Success SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 2

Introduction Printed Circuit Boards and Classification

• In today’s compressed development cycles • Printed circuit boards are the baseline in electronic packaging where rapid and cost-effective testing and – they are the interconnection medium upon which electronic analysis are key, a properly designed and components are formed into electronic systems. executed quality assurance function (with – PCB materials are generally glass reinforced organic appropriate reliability analysis) can enable polyimide (epoxy, BT, ceramic are also used). products with robust design margins.

• Classified on the basis of Examples of Bare PCBs – Dielectrics used Populated PCBA – Reinforcement • If the mission conditions are not well understood or the reliability analysis and – Circuit type accelerated testing are not conducted right, – Component types cost and schedule impacts, along with – Board construction unexpected failures will add risk to a – Design complexity Project development cycle.

SOURCE: Industrial Laser Solutions. PCBShop.org

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 3 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 4 Polyimide PCBA Supply Chain* Major Constituents of Laminates*

Glass Raw Materials Constituent Major function (s) Example material (s) (Silica, Limestone, Clay, Boric Acid) Provides mechanical strength and electrical Reinforcement Woven glass (E-grade) fiber properties Glass Fiber Production Polyimide Raw Materials (Formation, Coating/Binders, Yarns) (Petrochemical Derivatives) Bonds inorganic glass with organic resin and Coupling agent Organosilanes Raw materials suppliers transfers stresses across the structure E-Glass Plies/Fabrics Flame Retardants Prepregs/Cores Fillers and Additives Matrix Acts as a binder and load transferring agent Polyimide Laminate suppliers Copper Foil Laminates Dicyandiamide (DICY), Phenol Curing agent Enhances linear/cross polymerization in the resin Consumables (e.g., novolac (phenolic) Oxide Coatings Printed Circuit etchants, cleaners) Halogenated (TBBPA), Halogen- Drill Bits Board Panels Flame retardant Reduces flammability of the laminate Solder Mask/Silk Screen free (Phosphorous compounds) ENIG/HASL/ENEPIG w/Coupons Design and Coupon Data /OSP/other Plating Board fabricators Reduces dissipatation (high frequency), thermal Silica, Fillers expansion and cost of the laminate Aluminum hydroxide Active/Passive/Discrete AOI and Inspections Electronic parts, HW Assembly Increases reaction rate, reduces curing temperature, Imidazole, Accelerators Solder, flux, cleaning Processes ICT, Tests, Burn-in Assembly houses controls cross-link density Organophosphine chemistries * - Sood, Bhanu, and Michael Pecht. "Printed Circuit Board Laminates." Wiley Encyclopedia of Composites (2011). * - Sood, Bhanu, and Michael Pecht. "Printed Circuit Board Laminates." Wiley Encyclopedia of Composites (2011).

5 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 6

Example: Glass Fabric Treatment* Bare PCB Suppliers*

Glass Weave Style Glass Weave Style Glass Weave Style

1080 Style 2116 Style 7628 Style Fiber/resin interphase delamination occurs due poor glass treatment.

* - Sood, Bhanu, and Michael Pecht. "The effect of epoxy/glass interfaces on CAF failures in * - “Challenges and Opportunities: State of the U.S. Bare Printed Circuit Board Industry” Crawford M. and Botwin B., IPC APEX Expo, February 11-16, 2017, San printed circuit boards." Microelectronics Reliability (2017). Diego CA. Reproduced with permission.

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 7 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 8 Support to U.S. Government Agencies* Bare PCB Supplier Capabilities*

* - “Challenges and Opportunities: State of the U.S. Bare Printed Circuit Board Industry” Crawford M. and Botwin B., IPC APEX Expo, February 11-16, 2017, San * - “Challenges and Opportunities: State of the U.S. Bare Printed Circuit Board Industry” Crawford M. and Botwin B., IPC APEX Expo, February 11-16, 2017, San Diego CA. Reproduced with permission. Diego CA. Reproduced with permission.

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 9 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 10

Factors Causing PCB Production Material Supply Chain Disruptions* Bottlenecks*

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 11 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 12 PCB Quality Significance of Board Requirements

• In a vast majority of cases, NASA uses IPC standards (e.g., IPC-6012, 6013) • The requirements and coupons are a “front door”. – IPC-6012 for rigid, IPC-6013 flex, IPC-6018 high speed etc.. • Examples: • Inspection include: – Internal Annular Ring: • Egregious violations indicate there may have been a serious problem in – Microsection evaluation (coupons) development of the board (layup or lamination). – Surface finish evaluation (coupons) • Other NCs don’t indicate any risk at all (example: application of IPC- • Test include: 6012 Rev B. v/s IPC-6012 Rev. D) – External visual examination – Negative etchback v/s positive etchback: – Electrical continuity and isolation • Modern cleaning processes and flight experience result in equal reliability – Solderability (not 100% cases) with both etchback conditions or no etchback. – Cleanliness – Wicking of copper: • Requirements are conservative based on broad statistics. XRFSpectrum • In some cases MIL, ESA or “in- • A basic analysis of the board layout can indicate directly if there is risk or house” standards are applied. PTHinCrossͲ not, regardless of requirements violations. section

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 13 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 14

Study Objective

– Evaluate a subset of GSFC PCB suppliers (direct or indirect) and corresponding PCB coupon microsection testing data. – Develop a methodology for data generation and collection to provide trend analysis PCB Supplier Evaluation Study • Identifies/predicts violation of a process limit criteria (in case of an egregious NC). – Provide analysis for severity categories of the nonconformance. – Provide recommendations to the suppliers (i.e. supplier quality engineering, continuous process monitoring, quality metrics definition).

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 15 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 16 Requirements, Nonconformance, Data Microsectioning Generation and Collection IPC - PCB Multi-Issue Microsection Wall Poster* • Suppliers perform • Present study evaluates only the microsections performed by GSFC. microsectioning and inspect per specifications. – PCB coupon microsection evaluation in accordance to IPC • Secondary GSFC independent Standard (IPC-6018B Class 3, IPC-6012C Class 3/A). microsection analysis yielded – Coupon evaluation reports were generated, identified non- 20-30% inspection rejects, conformances. caused by: • All PCB coupon testing results from all GSFC suppliers were – Screening escapes: • Test sample quality not consistent recorded for the past 3 years (from 2015 – present) • Supplier microsection process, inadequate coupons – Data include nonconformance and conformances in accordance – Requirement interpretations with IPC Standards. – Requirements flow-down issues – Total number of data points are approximately 882 jobs. • Alternative specifications (MIL, ECSS) * - https://blog.ipc.org/2010/11/22/pcb-multi-issue- – Each job has number of nonconformance with different severity. • Buying heritage and off-the-shelf designs microsection-wall-poster/

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 17 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 18

Data Analysis –Submission and Nonconformance Study Methodology for Supplier • Since 2015, received and analyzed 882 PCB coupon submissions 0.3 638submissions from PCB suppliers. 0.25 0.2

• Top ten suppliers sent 638 submissions. 0.15

• Total nonconformance observed: 260 Fraction 0.1 0.05

0 12345678910 Top 10 Suppliers • For each supplier, analyzed nonconformance (s) Supplier'ssubmissionrate Nonconformancespread – Identify severity trend across top 10 GSFC suppliers by analyzing total submission by individual supplier Supplier submission rate = submission rate and nonconformance spread. total submission by all supplierݏ – Classifying and analyzing top 5 severity categories. total nonconformance by individual supplier Nonconformance spread = total nonconformance by all suppliers

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 19 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 20 Classification and Analysis - Top 5 Analyzing Top 5 Severities of Supplier’s Nonconformances Nonconformance

(A) Inner layer separations/inclusions Twenty one distinct conformances observed among the ten suppliers • Observations show the (D) Separation/inclusions between plating layers nonconformances with the NC Nonconformance Standard (E) Copper wicking in excess of 2.0 mil A Innerlayerseparations/inclusions IPC6012BClass3/A most occurrences (7 out of 10 B ElectrolessNilessthan118microinches IPC6012BClass3/A (F) Internal annular ring less than 2.0 mil CPlatingvoids IPC6012DS Suppliers) are D and F. D Separation/inclusionsbetweenplatinglayers IPC6012BClass3/A PCB Suppliers (J) ENIG is less than the minimum requirements E Copperwickinginexcessof2.0mil IPC6012BClass3/A F Internalannularringlessthan2.0mil IPC6012BClass3/A 12345678910 • Investigated the contributors note) IPC6012BClass3/A G Internalannularringlessthan5.0mil(drwg. H Externalannularringlessthan5.0mil IPC6012BClass3/A to implement techniques I Immersiongoldlessthan3.0microinches IPC6012DS A F E K A N EEA E Electrolessnickelandimmersiongoldplating which may eliminate theses J thickness<118microͲinches(Ni)and2microͲ IPC6012BClass3/A BGDFFOPA FF K Blindviaplatingthicknesslessthan0.8mil IPC6012BClass3/A nonconformances from at recessiongreatherthan3mil IPC6012BClass3/A LResin M Solidcoppermicroviavoidsinexcessof33% 8252313C CHBLDF CDST least 7 suppliers. NLaminatedelamination IPC6012BClass3/A Olaminatecracks IPC6012CClass3/A D A IJJE D F DU P Etchbacklessthan0.2mil IPC6012BClass3/A Q Immersiongoldplatingthicknessinexcessof6mil IPC6012CClass3/A than1.0mil IPC6012BClass3/A R Copperplatingthicknessless suppliers from Nonconformances Common E DJA MPQRPR SLaminatecrackgreaterthan3.0mil IPC6012BClass3/A T Dielectricthicknesslessthan3.0milmin IPC6012BClass3/A ULaminatevoidgreaterthan3.0mil IPC6012BClass3/A * - “Challenges and Opportunities: State of the U.S. Bare Printed Circuit Board Industry” Crawford M. and Botwin B., IPC APEX Expo, February 11-16, 2017, San Diego CA. Reproduced with permission.

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 21 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 22

Inner Layer Separations or Inclusions Inner Layer Separations or Inclusions

• Separation of inner-layer foil and the plated through hole barrel. Contributors Resolution • Inclusion - contaminant material that is • Improper lamination press or cure cycles • Consistency in drilling processes. present in an area where it is not whether it be pressure, time, temperature. • Reduce the resin content in the expected. • Others include inadequate coverage of stack up. inner layer oxide, moisture not • Good desmear, with adequate texture. Risk: intermittent electrical open or completely removed in pre-lamination • Provide adequate copper border complete open after board is bake cycle. subjected to thermal excursions for support and resin venting (reflow, wave soldering or rework) • Bad batch of prepreg and or laminate. • Post-electroless copper cleaning residues, contaminated pretreatment prior to 1. IPC-6012 – Qualification and Performance Specification for Rigid Printed Boards. electrolytic plating, or an out-of-control 2. Swirbel, Tom, Adolph Naujoks, and Mike Watkins. "Electrical design and simulation of high density printed circuit boards." IEEE transactions on advanced packaging 22.3 (1999): 416-423. electrolytic copper process.

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 23 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 24 Separation or Inclusions Between Plating Layers Separation or Inclusions Between Plating Layers

Plating separation -The separation between a plating layer and foil. Contributors Resolution • Incomplete wrap plating • Adjust plating parameters • Overly-aggressive cleaning • Optimize cleaning processes process • Insufficient cleaning

Risk: intermittent electrical open or complete opens due to mechanical or thermal stresses.

1. IPC-6012 – Qualification and Performance Specification for Rigid Printed Boards. 2. Yung, Edward K., Lubomyr T. Romankiw, and Richard C. Alkire. "Plating of Copper into ThroughǦHoles and Vias." Journal of the Electrochemical Society 136.1 (1989): 206-215.

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 25 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 26

Copper Wicking in Excess of 2.0 mil Copper Wicking in Excess of 2.0 mil

The extension of copper from a PTH along the glass fiber fabric. Contributors Resolution • Dull drill bits or broken drill bits • Optimize desmear parameters that causes a crack in the • Improve drilling operation (feed laminate. and speed). Risk: intermittent electrical shorts or • Incompatible laminate material • Ensure sufficient resin wet-out of complete shorts due to bias driven • Insufficient glass etch. glass fibers (siloxane treatment). migration of copper towards non- • Poor glass to organic adhesion. common conductors.

1. Sood, Bhanu, and Michael Pecht. "Printed Circuit Board Laminates." Wiley Encyclopedia of Composites (2011). 2. Tummala, Rao R., Eugene J. Rymaszewski, and Y. C. Lee. "Microelectronics packaging handbook." (1989): 241- 242. 3. IPC-6012 – Qualification and Performance Specification for Rigid Printed Boards.

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 27 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 28 Internal Annular Ring Less Than 2.0 mil Internal Annular Ring Less Than 2.0 mil

This occurs, when the inner layer copper pad (measured from the hole wall plating to Contributors Resolution its outer most length) is less than 2 mils. • Drilled-hole pattern not matching • Better material selection of the lands on the internal layers laminate, improved cleanliness, (Misregistration). and reduction in the amount of • Lamination process. volatiles. Risk: inner layer breakouts after the • Prelamination treatments that • Confirm whether or not it is board is subjected to thermal involve scrubbing or bending may operator error. excursions (reflow, wave soldering or stretch the thin laminate, which • Update drawing notes to bring the rework) leading to intermittent will then shrink after it is etched notes in line with current industry electrical or complete open behavior. and baked dry. maturity levels. • Application of specification or 1. Sood, Bhanu, and Sindjui, N. "A Comparison of Registration Errors Amongst Suppliers of Printed Circuit Boards“, drawing notes. Proceedings, IPC APEX Expo (2018). 2. IPC-6012 – Qualification and Performance Specification for Rigid Printed Boards.

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 29 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 30

ENIG (Au or Ni) Less than the Minimum ENIG Less than Minimum

Electroless nickel and/or immersion gold XRFSpectrum plating thickness (ENIG) is less than the Contributors Resolution minimum requirements (118 micro-inches • Improper cleaning of surfaces. • Re-clean copper using chemical for Ni and 2 micro-inches for Au). cleaners or mechanical • Improper or inadequate rinsing. • scrubbing Institute micro-etch step Risk: (1) solderability and, (2) • Bath parameters not being to improve cleaning excessive dissolution of copper into followed (pH and chemical). • Improve rinsing( Check flow, the bulk solder (forming brittle • Bath temperature too low. agitation and water quality) intermetallic) when nickel is thin. • Copper surface not clean of oil or • Raise temperature per supplier inhibiting film. specifications 1. Johal, Kuldip, and Jerry Brewer. "Are you in control of your electroless nickel/immersion gold process?." Proc. Of • Readjust to supplier operational IPC Works. No. S03-3. 2000. parameters 2. Meng, Chong Kam, Tamil Selvy Selvamuniandy, and Charan Gurumurthy. "Discoloration related failure mechanism and its root cause in Electroless Nickel Immersion Gold (ENIG) Pad metallurgical surface finish." Physical and Failure Analysis of Integrated Circuits, 2004. IPFA 2004. Proceedings of the 11th International Symposium on the. IEEE, 2004. 3. IPC-4552 – Specification for Electroless Nickel/Immersion Gold (ENIG) Plating for Printed Circuit Boards

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 31 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 32 Summary of Supplier Study

• The test data is analyzed using statistical method to provide trend analysis for all suppliers. – Root cause(s) and key contributors are identified. – Mitigation plan is included for the root cause of nonconformance. New technology Implementation: Technology Readiness Levels • Provide recommendations to the supplier’s process, identification and prediction of nonconforming process limit criterion, and to improve test standards. • New technologies (example: smaller annular rings, via-in-pads, thinner laminates or newer plating) are implemented on the basis of supplier maturity and reported NCs.

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 33 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 34

Technology Readiness Levels TRL Implementation – Considerations

“TRLs are a set of metrics that enable the assessment of the maturity of a particular • A new technology can be at a different technology and the consistent comparison of the maturity between different types TRL depending on the requirements. of technology in the context of a specific application, implementation, and • Not all new designs are new operational environment.” technology – Some may be considered “standard OnceTLR6is engineering” (e.g., a new primary demonstrated,therisk structure based on existing design associatedwiththenew technologyisroughly and fabrication processes) equivalenttotheriskofa newdesignthatemploys • The configuration for TRL verification occurs at the lowest level of integration that exhibits standardengineering the new performance/functionality. practiceandisbounded bypreviously • The “weakest link” approach is used to determine the TRL of a subsystem implementedgroundͲ – There can be cases where a subsystem’s TRL is lower than that of all of its elements basedsystems. (e.g., a new architecture that is used to provide new performance, but employs all “heritage parts”).

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 35 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 36 Risk

Risk is an expectation of loss in statistical terms.

Definition: the combination of a) the probability (qualitative or quantitative) that an undesired event will occur, and Risk Based Technology Evaluation and Insertion b) the consequence or impact of the undesired event

• Flavors of risk (consequences) Communicating risk – Technical (failure or performance degradation on- is key to portraying orbit) the status of a new – Cost ($ it will take to fix the problem) technology and – Schedule (time to fix the problem) project in – Safety (injury, death, or collateral damage) development.

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 37 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 38

Balanced Risk - Maintaining a Level Risk vs. Possibility Waterbed* • Failure modes and mechanisms can appear through – Analysis and simulation A systems approach of looking across all options to ensure that mitigating or – Observation eliminating a particular risk does not cause much greater risk somewhere in the – Prior experiences system. – Brainstorming “what if” scenarios Trytomaintainthelevelwaterbed – Speculation • These all constitute possibilities

• There is a tendency to take action to eliminate severe Pushingtoohardonindividualriskscancause consequences regardless of the probability of occurrence otherriskstobeinordinatelyhigh • When a possibility is combined with an environment, an operating regime, and supporting data, a risk can be established—this is core to the engineering process.

• Lack of careful and reasoned analysis of each possibility in terms of the conditions that results in the consequence and the probability of occurrence will result in excessive cost and may increase the overall risk. * - Leitner, J., Sood, B., Isaac, E., Shue, J., Lindsey, N., & Plante, J. (2018). Risk-Based Safety and Mission Assurance: Approach and Experiences in Practice. Quality Engineering, (just-accepted), 1-40.

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 39 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 40 Impact of Non-conformances Risk Assessment

• Bare boards cost $$ and build • Traceable PCB test coupons (designed per specs. such as IPC-2221B) are schedules – expensive!! submitted to GSFC or to a GSFC-assessed laboratory. • But failures are even more expensive! • Reports that indicate nonconformance are dispositioned by risk assessment • Test sample nonconformance is not performed prior to refabricating or populating the PCB. the same as PCB failure. – If risk assessment indicates elevated risk due to the • Risk-based decisions are used for nonconformance, then use is dispositioned by MRB. disposition of non-conformances. • Risk assessment process eliminates waste and saves money and schedule, • Non-conformances may have little to lowers overall risk for the project. no impact per application. • The process reduces the need for repeated attempts to refabricate. • Began to explore origins and merit of requirements (more later).

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 41 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 42

Summary Acknowledgements

Lessons learned Supplier capability TRL Evaluation from non- and assessment conformance data NASA Workmanship NASA Office of Safety Risk Assessment for Program and Mission Assurance, new technology Quality Program insertion

• Risk-based new technology assessment centered around understanding all sides of NASA Goddard Risk risk. and Reliability Branch • Lessons learned are at the core of the methodology • This approach is effective at saving cost and schedule resources. • Enables any project to operate at the lowest possible risk posture given its particular resource constraints.

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 43 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 44 Backup Slides

Bhanu Sood Commodity Risk Assessment Engineer Risk and Reliability Branch Quality and Reliability Division NASA Goddard Space Flight Center

+1 (301) 286 5584 [email protected]

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 45 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 46

TRL Definition and Decomposition by Fidelity of Build Factor*

TRL DefinitionfromNPR CompletionCriteriafrom Mission Performance/Function FidelityofAnalysis FidelityofBuild LevelofIntegration EnvironmentVerification 7123.1e NPR7123.1e Req. Table3.1.6Ͳ1:FidelityofBuild 4 Componentand/or Documentedtest Generic Basicfunctionality/ Mediumfidelity:topredictkey Lowfidelity:breadͲ Component/Assemb Testedinlaboratoryfor breadͲboardvalidated performance classof performance performanceparametersandlife board ly criticalenvironments Performance/ Environmental inlaboratory demonstratingagreementmissions demonstrated limitingfactorsasafunctionof Unit Purpose FormandFit/Scaling Pedigree environment withanalytical relevantenvironments Relevantenvironments Function Requirements predictions.Documented identified.LifeͲlimiting definitionofrelevant mechanismsidentified. Breadboard ProofͲofͲconceptforapotentialdesign Demonstrate Notrequired,e.g.laidout Testedinalaboratory NA

environment. performance/function flatonlabtable environment Documentedtest Genericor Mediumfidelity:topredictkey 5 Componentand/or Basicfunctionality/ Mediumfidelity: Component/ Testedinrelevant Brassboard Demonstratefeasibilityofformandfit, Demonstrate Approximate(notflat) Designedtomeetrelevant NA brassͲboardvalidatedin performance specific performancemaintained performanceparametersandlife brassͲboardwith Assembly environments environments performance/function withscalingfactors environmental relevantenvironment demonstratingagreementclassof limitingfactorsasafunctionof realisticsupport Characterizephysicsof understood requirements withanalytical missions relevantenvironments Technology elements lifeͲlimitingmechanisms predictions.Documented andfailuremodes. Development Prototype Representativedesign;pathfinder;demonstrator Testedtomeet Representativewith Testedtomeetrelevant NA,butmaybe definitionofscaling New performance/function scalingfactorsunderstood environmental partialorfull requirements. requirements requirements 6 System/subsystem Documentedtest Specific Requiredfunctionality/ Mediumfidelity:topredictkey Highfidelity: Subsystem/System Testedinrelevant Finalizedetaileddesign Testedtomeet Exactasknownattimeof Testedtomeetrelevant NA,butmaybe modelorprototype performance mission performance performanceparametersandlife prototypethat environments.Verifyby Engineering demonstratedina demonstratingagreement demonstrated limitingfactorsasafunctionof addressesallcritical testthatthetechnologyis Unit performance/function build environmental partialorfull relevantenvironment withanalyticalpredictions operationalenvironments scalingissues resilienttotheeffectsof requirements requirements lifeͲlimitingmechanisms Qualification Qualifydesign Testedtomeet Exactasknownattimeof Testedtomeetflight Full 7 Systemprototype Documentedtest TechͲnology Requiredfunctionality/ Highfidelity:topredictkey HighFidelity: Subsystem/System Testedinactual Unit performance/function build qualification demonstrationinan performance demonͲ performance performanceparametersandlife prototypeor operationalenvironment requirements environmental demonstratingagreementstration limitingfactorsasafunctionof operational demonstrated engineeringunitthat requirements environment withanalyticalpredictionsmission operationalenvironments addressesallcritical Development scalingissues FlightUnit FinalProduct Testedtomeet Exact Testedtomeetflight Full 8 Actualsystem Documentedtest Specific Requiredfunctionality/ Highfidelity:topredictkey Finalproduct: System Testedinproject performance/function qualification completedand“flight performanceverifying mission performance performanceparametersandlife environmentalverification requirements environmental qualified”throughtest requirementsand demonstrated limitingfactorsasafunctionof Flightunit; program. requirements

anddemonstration analyticalpredictions operationalenvironments Engineering Lifetestunitforlife Completedlifetests. FlightSpare FinalProduct Testedtomeet Exact Testedtomeetflight Full limiteditems* performance/function qualification 9 Actualsystemflight Documentedmission Specific Requiredfunctionality/ Highfidelity:topredictkey Finalproduct: System Operatedinactual requirements environmental proventhrough operationalresults mission performance performanceparametersandlife operationalenvironment requirements successfulmission verifyingrequirements demonstrated limitingfactorsasafunctionof Flightunit operations operationalenvironments NASA Systems Engineering Processes and Requirements (NPR) 7123.1B (Table 3.1.3-1) SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 47 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 48 TRL 6 Risk Classification (NPR 7120.5 Projects)

At the Subsystem level • Class A: Lowest risk posture by design – Failure would have extreme consequences to public safety or high priority national science objectives. • Specific mission (and specific mission risk class) – In some cases, the extreme complexity and magnitude of development will result in a system launching with many low to medium risks based on problems and anomalies that could not be completely resolved under cost • Required functionality/ performance demonstrated and schedule constraints. – Examples: HST and JWST • Medium fidelity: to predict key performance parameters and life • Class B: Low risk posture limiting factors as a function of operational environments – Represents a high priority National asset whose loss would constitute a high impact to public safety or national science objectives • High fidelity: prototype that addresses all critical scaling issues – Examples: GOES-R, TDRS-K/L/M, MAVEN, JPSS, and OSIRIS-REX • Class C: Moderate risk posture • Subsystem tested in relevant environments. – Represents an instrument or spacecraft whose loss would result in a loss or delay of some key national – Verify by test that the technology is resilient to the effects of life- science objectives. – Examples: LRO, MMS, TESS, and ICON limiting mechanisms • Class D: Cost/schedule are equal or greater considerations compared to mission success risks – Technical risk is medium by design (may be dominated by yellow risks). – Note, “relevant environment” is a subset of the operational – Many credible mission failure mechanisms may exist. A failure to meet Level 1 requirements prior to minimum environment and specifically focuses on “stressing” the new lifetime would be treated as a mishap. technology – Examples: LADEE, IRIS, NICER, and DSCOVR

SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 49 SAFETY and MISSION ASSURANCE DIRECTORATE Code 300 50