Jet Propulsion Laboratory California Ins tute of Technology
Mul core in Space NASA and AFRL Invest in the Future of Flight Compu ng
Richard Doyle and Raphael Some Jet Propulsion Laboratory, California Ins tute of Technology
Gabriel Mounce, Air Force Research Laboratory, Kirtland AFB
Wesley Powell, NASA Goddard Space Flight Center
Montgomery Goforth, NASA Johnson Space Center
Stephen Horan, NASA Langley Research Center
Michael Lowry, NASA Ames Research Center
presented at Future In-Space Opera ons Working Group September 17, 2014
© 2014 California Ins tute of Technology. Government sponsorship acknowledged. The NASA Need There are important mission scenarios that today cannot be accomplished robustly and cost-effec vely • Space-based compu ng has not kept up with the needs of current and future NASA missions • Government and industry are developing high- performance space-qualifiable processors • NASA con nues to have unique requirements – Deep space, long dura on, robo c and human missions – Higher performance, smaller spacecra and lower cost • Onboard science data processing • Autonomous opera ons and greater resilience – Extreme needs for low power and energy management, efficiency, fault tolerance and resilience
9/16/14 2 NASA Space Technology Mission Directorate The Game Changing Development Program (GCDP) commissioned a High Performance Spaceflight Compu ng (HPSC) Study
Iden fy Driving Capture Applica ons Desirements (Human) (HEOMD) Derive Requirements Perform Trade: Iden fy Driving Capture Applica ons Desirements Architectures vs. (Robo c) (SMD) KPPs
Perform Gap Analysis Derive Results Review and Evaluate each Analyze Architecture for Compu ng each Key Make Architectures Performance Recommenda ons Parameter (KPP) 9/16/14 3 HPSC Formula on Phase Study Summary General-purpose mul -core provides op mum ROI for NASA
The Assignment The Results
Iden fy relevant NASA use cases Developed 9 human spaceflight (HEOMD) and 10 science mission (SMD) What are the paradigm-shi ing NASA space-based use cases for future flight compu ng, spanning cri cal mission applica ons that will drive next genera on flight func ons, high data rate instruments, and autonomy u lizing model- compu ng? based reasoning techniques Derive requirements 100X performance increase, low power (down to 7W) with scaling, What are the future onboard compu ng support for a range of fault tolerance, common programming requirements? languages, avoidance of addi onal V&V effort, interoperable with co- processors Perform a gap analysis No exis ng or emerging spaceflight processors possess all necessary How/where do commercial and defense industry performance, power efficiency, reliability, and programmability developments in compu ng fall short of NASA’s unique a ributes requirements and architectural needs? Trade architectures against defined Key Performance Rad-hard general-purpose mul -core best addresses the future flight Parameters (KPPs) compu ng requirements and presents the most affordable gap against Which compu ng architecture will make the most the KPPs difference? Make a recommenda on Competed/directed program plan for rad-hard general-purpose mul - How can NASA best invest limited resources to meet core, with solu ons for power/energy, fault tolerance and other NASA the future needs of its space systems? requirements, leveraging other agency and industry investments
9/16/14 Specifics appear in the following charts 4 NASA Applica ons for High Performance Spaceflight Compu ng Human Spaceflight Science Mission (HEOMD) Use Cases (SMD) Use Cases 1. Cloud Services 1. Extreme Terrain Landing 2. Advanced Vehicle Health 2. Proximity Opera ons / Forma on Management Flying 3. Crew Knowledge Augmenta on 3. Fast Traverse Systems 4. New Surface Mobility Methods 4. Improved Displays and Controls 5. Imaging Spectrometers 5. Augmented Reality for 6. Radar Recogni on and Cataloging 7. Low Latency Products for Disaster 6. Tele-Presence Response 7. Autonomous & Tele-Robo c 8. Space Weather Construc on 9. Science Event Detec on and 8. Automated Guidance, Naviga on, Response and Control (GNC) 10. Immersive Environments for 9. Human Movement Assist Science Ops / Outreach High value and mission cri cal applica ons iden fied by 9/16/14 NASA scien sts and engineers 5 Science Mission Applica ons 10X improvement for exis ng applica ons Enables new science and mission capabili es on future missions
SMD Use Cases Benefits to Missions 1. Extreme Terrain Landing • Extreme Terrain Landing 2. Proximity Opera ons / – Enables reliable and safe landing in Forma on Flying hazardous terrain: TRN and HDA 3. Fast Traverse algorithms benchmarked by Mars 4. New Surface Mobility Methods Program - required six (6) dedicated RAD750s 5. Imaging Spectrometers 6. Radar • Fast Traverse 7. Low Latency Products for – Remove computa on as a limi ng factor Disaster Response to mobility – drive 10X faster and more, safely (wheel slip, obstacle detec on) 8. Space Weather 9. Science Event Detec on and • Science Event Detec on and Response Response 10. Immersive Environments for – Increase capture rate for dynamic, Science Ops / Outreach transient events from ~10% to >75%, with <5% false posi ves, for increased and more mely science return
No longer need to size science / mission scope 9/16/14 to flight compu ng capability 6 Deriving Requirements Requirements Template
• Space Environment(s) – Radia on environment at the me of applica on; e.g., geosynchronous (GEO), low-Earth orbit (LEO), deep space? • Spacecra Power Environment(s) / Constraint(s) – Available power for avionics and compu ng, e.g., small spacecra or rover with limited power availability (6 Wa s processor power, 10-15 Wa s computer power), medium sized spacecra (7-12 Wa s processor power, 15-30 Wa s computer power), or large spacecra with large power budget (>12-25 Wa s processor power, >30-100 Wa s computer power)? • Cri cality/Fault Tolerance – Is this applica on life or mission cri cal, must it operate through faults, can it tolerate errors if detected? • Real-Time – Does the applica on have a hard real- me deadline; if so, what is the required ming? • Type(s) of Processing – Algorithm kernel(s). Is it primarily e.g., digital signal processing (DSP), data base query, is it parallelizable, is it amenable to a data flow approach? • Memory Access Pa ern – What is the primary data movement pa ern, e.g., does it fetch data from memory once and then flow through a processing chain, or does it access data in a con nuous random access pa ern, or does it access sequen al data in a con nuous and regular pa ern? • Duty Cycle – What is the pa ern of execu on, e.g., is the applica on called con nuously over a long period of me, is it called once and operate for only a short dura on, is the applica on execu on profile spiky and/or unpredictable? • Data Rate 9/16/14 – What are the I/O and memory access data rates? 7 NASA Flight Compu ng High-Level Requirements as derived from the NASA use cases
Computa on Mission Need Objec ve of Flight Architecture Processor Type and Category Computa on A ribute Requirements
Vision-based • Terrain Rela ve • Conduct safe proximity • Severe fault tolerance • Hard real me / mission cri cal Algorithms Naviga on (TRN) opera ons around and real- me • Con nuous digital signal processing with Real-Time • Hazard Avoidance primi ve bodies requirements (DSP) + sequen al control processing Requirements • Entry, Descent & • Land safely and • Fail-opera onal (fault protec on) Landing (EDL) accurately • High peak power • High I/O rate • Pinpoint Landing • Achieve robust results needs • Irregular memory use within available • General-purpose (GP) processor meframe as input to (10’s – 100’s GFLOPS) + high I/O rate, control decisions augmented by co-processor(s) Model-Based • Mission planning, • Con ngency planning to • High computa onal • So real me / cri cal Reasoning scheduling & mi gate execu on complexity • Heuris c search, data base Techniques for resource failures • Graceful degrada on opera ons, Bayesian inference Autonomy management • Detect, diagnose and • Memory usage (data • Extreme intensive & irregular • Fault management recover from faults movement) impacts memory use (mul -GB/s) in uncertain energy management • > 1GOPS GP processor arrays with environments low latency interconnect High Rate High resolu on • Downlink images and • Distributed, dedicated • So real me Instrument sensors, e.g., SAR, products rather than raw processors at sensors • DSP/Vector processing with Data Hyper-spectral data • Less stringent fault 10-100’s GOPS (high data flow) Processing • Opportunis c science tolerance • GP array (10-100’s GFLOPS) required for feature ID / triage
9/16/14 Future NASA use cases require drama c improvement over RAD750 (~400 MOPS) 8 Mapping of Use Cases to the Computa on Categories
Crew Knowledge Extreme Terrain Landing Improved Displays & Augmenta on Systems Controls Automated Guidance, Immersive Environments Vision Based Algorithms with Augmented reality for Nav & Controls (GNC) for Science Ops Outreach Real Time Requirements Recogni on & Cataloging Human Movement Assist Fast Transverse
Autonomous Mission Planning Advanced Vehicle Health Management Proximity Opera ons / Model Based Reasoning Autonomous & Tele- Forma on Flying Techniques Robo c Construc on New Surface Mobility Methods
Imaging Spectrometers Tele-Presence Low Latency Products for High Rate Instrument Data Radar Disaster Response Processing Cloud Services Space Weather 9 HEOMD SMD Compu ng Architectures Candidates evaluated under the HPSC task
• General-purpose mul -core – Rad-hardened – COTS • DSP mul -core – Rad-hardened – COTS • Reconfigurable compu ng (e.g., FPGAs) – Rad-hardened – COTS • Graphics processing units (GPUs) – Rad-hardened – COTS • Also: Hybrid architectures u lizing co-processors – General Purpose Mul core + Reconfigurable Compu ng – General Purpose Mul core + GPU – General Purpose Mul core + DSP Mul core 9/16/14 10 Key Performance Parameters
Applica on-referenced KPPs – Computa onal performance – Radia on and fault tolerance – Power and energy management – So ware verfica on and valida on Architecture-referenced KPPs – So ware verifica on and valida on (this is the single cross-over KPP) – Programmability and flight so ware applicability – Interoperability – Extensibility and evolveability Addi onal KPPs – Non-recurring cost – Recurring cost – Cross-cu ng applicability across the NASA mission set
Other Constraints
– TRL 5/6 in <=3 years within available resources 11 9/16/14 Evalua on of the Compu ng Architectures
Key Performance Rad-hard Parameters Rad-hard Rad-hard Graphics (KPPs) – General Purpose Rad-hard DSP Reconfigurable COTS-based Processing Scoring Multicore Multicore Computing Multicore Units
Total KPP 1 5 2 3 4 score (rank)
#KPP scores 12/12 4/12 7/12 6/12 5/12 above mean
9/16/14 12 Study Recommenda on Rad-hard General Purpose Mul -core
Rad-hard General Purpose Mul core • Best overall fit to applica on requirements 3000 – Provides both general purpose and some DSP BEST Existence Proof – (44 GOPS) capability as well as interoperability with co- 2500 processors (DSP, reconfigurable) 2000 • Conducive to Power Scaling at core-level – Power Dissipa on issues to address fit within 1500 SOA available investment resource envelope 1000 • Conducive to thread-based Fault Tolerance 500 SOP
– Fault detec on/correc on/isola on Power Efficiency (MOPS/W) RAD750 (0.2 GOPS) WORST – Ability to segregate failed cores from the pool of 0 available cores in support of graceful degrada on 0 5 10 15 20 25 • Scalable to 10x increase in the number of cores Power (W) – Combined with power scaling, allows the increased Performance cores to consume power only as thread-load (GOPS) dictates
Computa onal Performance, Efficiency, and Scalability of Mul -core 9/16/14 redefines general-purpose compu ng for space systems 13 HPSC Formula on Task Recommenda ons Investment Focus and Approach
• Focus on Rad-hard General-purpose Mul -core – Leverage government and industry investments • Issue a BAA for hardware architecture designs in FY13 – Solicit flight compu ng system concepts – Prepare NASA requirements and benchmarks for early evalua on of architectures – A compe ve ini al phase, seeking innova ve solu ons and early risk re rement • Product of the investment – Mul -core hardware chip with bundled real- me opera ng system (RTOS) and FSW development environment, integrated on an evalua on board • Include a directed so ware investment – Middleware elements for alloca ng/managing cores for varying opera onal objec ves, working closely with the FSW community, driven by knowledge of the NASA applica ons
Challenge the community to develop an innova ve, extremely high performance, low power, flexible, rad-hard GP mul -core processor within available budget and schedule
9/16/14 Enter AFRL! 14 Future Flight Compu ng AFRL vs. NASA Requirements
Venn Diagram Next Generation Space Processor NASA vs AFRL AFRL NASA
Segregation and Routing around failed/depowered cores
Performance Throughput Memory Power Scalability Radiation Hardened Memory Scheme and Interface Core Power Scalability Radiation Tolerant Replication Based F.T. Fault Tolerant Interconnects Middleware Software Strategy Interconnect Strategy
There was sufficient common ground to jus fy a joint AFRL-NASA program 9/16/14 15 Joint NASA-AFRL Program Concept Next Genera on Space Processor (NGSP)
FY13 FY14 FY15 FY16 FY17 FY18
AFRL Development Phase
NASA
NASA
Ini al joint program plan was approved and funding allocated for Innova on Phase BAA issued, vendors selected, and benchmark development ini ated 9/16/14 16 NGSP / HPSC Program Innova on Phase – Top Level Objec ves Summary
• 24 Cores providing 20 GOPS/10GFLOPS, mul ple 10Gb/s I/O and DDR 3 Memory Ports at 7 Wa s • Dynamically power scalable at core level granularity by powering and depowering cores in real me without disrup ng system opera on, with very low idle power load (<<1W) • Fault tolerant interconnects between cores and to external I/O and memory devices • Built in self test with faulty core bypass and power up into known good state • Na vely supports N-Modular Redundancy where N=2, 3 • Segrega on and rou ng around failed and depowered cores • Radia on tolerant to 1MRad, Latch-up Immune, with Single Event Upset (SEU) rate of not greater than 0.01 event/day in Adams 90% worst case GEO environment • Interoperable with other high performance compu ng architectures, e.g., reconfigurable compu ng FPGAs • Real me support • System so ware and applica on development environment
**Proposers were encouraged to offer alterna ves ** 9/16/14 17 NGSP / HPSC Program Selected Vendors
Objec ve (HPSC Innova on Phase) • Reduce technical risk via the development of mul ple hardware architecture designs for a rad-hard general-purpose mul core flight compu ng system • Evaluate those designs against a set of benchmarks representa ve of the future NASA applica ons
Selected Vendors • Vendor Kick-Off mee ng held at Aerospace Corp, Nov 6-7, 2013 • Briefings provided by the three selected vendors • BAE Systems (Manassas, VA) • Boeing (Sea le, WA) • Honeywell (Clearwater, FL) • A ended on government side by • NASA (GCDP, JPL, GSFC) • USAF (AFRL, SMC, Aerospace) Kick-Off Mee ng Summary: Shown: MAESTRO, Rad-hard by • Official start to effort to finalize USAF use cases / requirements design (RHBD) 7X7 led mul core • To be reintegrated with NASA requirements by March 2014 chip (proof-of-concept for space- • NASA benchmark ac vity, with technical and programma c objec ves, qualified mul core compu ng) described to vendors 18 • Early insight on architectural approaches being taken by vendors NGSP / HPSC Program Use of Benchmarks
• Without benchmarking, there is considerable risk that a computer will not live up to its expected performance • Prospec ve users typically assess performance based on processor clock rate, word width, and instruc on set • However, design features can introduce bo lenecks that can limit performance for real-world applica ons to a frac on of the expected performance • O en these bo lenecks are difficult to iden fy via design reviews • Benchmarks provide the most reliable means to assess real-world performance • Case in point – RAD750 • Original processor had serious performance limita ons due to memory performance • Modifica ons were necessary in the implementa on of the L2 cache and DMA • The risk may be greater • For mul -core processors, because applica on performance can be sensi ve to architecture (i.e., topology, interconnect methodology, memory implementa on), it is essen al that the op mal architecture be established up front for the broad array of space-based applica ons
19 NGSP / HPSC – Development Phase Alignment and Infusion Strategy 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022
Board-level Flight Compu ng HPSC (NASA) $1M $13M Product Multi-core Processor Board Non-Flt Ethernet UART If dual port DDR/DDR2 is available, should consider I/F connecting the Actel FPGA to the DDR/DDR2 for optimal I/F data transfer to/from the host bus 10/100/1G RS232 DDDDRR/D/DDDRR22 ( s(sizizee??)) Companion Board PHY XCVR DDRD/RD/DRD2R 24 G(sbizyete?s)
s • e Rad-hard general-purpose mul core chip c a f
r Core Ref Clk e
s PLL t t n I
n
c DDR Ref Clk i c e
f PLL i i f i c OSC c m e
c XAUI XAUI e i XAUI XAUI XAUI Ref Clk p u p g PLL S r Maestro o
S t
L n p s
o Processor GBE Ref Clk i p t Cfg PLL n A a I c
i I/F l
p Clock Generation Logic p XAUI • A Virtex5 Real- me opera ng system Innova on Phase Development Phase CPU_RST Housekeeping TLM Ckt
POR CLK_ENA XAUI TLM I/F WD_TO • SDRAM Flight so ware development 5.0V ASIC 128M x 48bits 3.3V 2.5V V5 Cfg WD Mem L1 Boot Ctlr TImer Ctlr Code (x3) 1.8V Virtex5 Cfg 1.2V Host Bus Bridge File 1.0V (With DMA capability) NGSP (AFRL) Power Dist Unit NOR Flash $1M $9M 32M x 48 bits environment Host Bus • Middleware for core management, op mized for NASA applica ons
Testbed Evalua on A promising area of discussion is a technology AAE Testbed Mars 2020 payload concept for Mars 2020 to provide HPSC Technology The Advanced Explora on Systems (AES) Avionic Architectures for capability for landed func ons, which would be phased in gradually over the course of the surface Payload Explora on (AAE) project is considering the use of a SW simula on during 2014 and an FPGA implementa on in late 2015. The AAE mission. A spare board slot on the Rover Compute testbed would also evaluate an HPSC implementa on as it becomes Element (RCE) is available but needs to be available. reserved soon.
Fast Surface Mission Traverse Science Infusion The HPSC team is engaged with NASA SMD and HEOMD mission customers on the use cases documented in the HPSC Formula on Phase Study Report. Via the partnership with AFRL, a similar engagement is occurring with SMC for USAF customers. In addi on, HPSC team members are engaging with other poten al na onal-level customers and programma c partners (e.g., NRL/USN). Human- 20 AFRL High Data Rate Robo c Qualifica on Instruments Teaming NGSP / HPSC Program Product Descrip on
• Rad-Hard, General-Purpose, Mul core Processor – Enabling new flight capabili es and missions for the next 15+ years • 100X computa onal performance / efficiency vs. RAD750 – MIPS to 10s of GOPS/GFLOPS at <<1Wa to 7Wa s • Extensible by cascading, ling, paralleling – For performance and/or fault tolerance • Dynamically configurable over power/performance/fault tolerance space • Applicable from Cubesats/Smallsats to flagship missions to 3000 crewed missions BEST Existence Proof – (44 GOPS) – Available at chip level as well as board level 2500 2000 • System So ware, Run me and Development Environments – OS, Hypervisor, Debugger, C/C++ 1500 SOA 1000 • Middleware Layer – Dynamic System Configura on 500 SOP
– Resource Allocator, Power Manager, Fault Tolerance Manager Power Efficiency (MOPS/W) RAD750 (0.2 GOPS) WORST 0 0 5 10 15 20 25 • Evalua on board and reference design kit Power (W)
Performance (GOPS) 21 Summary
• Future NASA mission scenarios call out for significantly improved flight compu ng capability • Improved flight compu ng means enhanced computa onal performance, energy efficiency, and fault tolerance • AFRL independently iden fies the need for improved flight compu ng • The NGSP program was ini ated to address this joint need • NASA and AFRL have defined a set of objec ves to guide the defini on of processor architectures • Benchmarks have been developed to evaluate those architectures against candidate use cases • The study results will guide a Development Phase to follow
It is me to move beyond the 1990’s technology of the RAD750 and 9/16/14 Redefine the role of compu ng in space systems 22 Acknowledgements
• This research was carried out in part at the Jet Propulsion Laboratory, California Ins tute of Technology, under a contract with the Na onal Aeronau cs and Space Administra on (NASA), funded by the Game Changing Development Program of the NASA Space Technology Mission Directorate.
9/16/14 23