Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center

Ground System Architectures Workshops (GSAW2004) Manhattan Beach, California March 30, 2004

Lori Enright, NASA/Goddard Space Flight Center Charlie Bengston, Ronald A. Jones/ASRC Aerospace Project Background

„ Budgets for NASA space missions typically extend through the end of their original design lifetime

„ Scientifically valuable missions which keep operating into an “extended mission phase” are faced with severe operating cost limitations

„ TRMM was launched in 1997 with a planned operational life of 3 years - our desire is to extend the mission through 2011

„ For TRMM, the NASA Earth Science Mission Operations (ESMO) Office is creating a “pathfinder” system based on GSFC’s plug-and- play message-oriented GMSEC architecture

GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 2 NASA’S Tropical Rainfall Measuring Mission

„ Launched in Nov 1997 with a design lifetime of 3 years

„ Low earth orbit: 402 km, 35 degree inclination

„ Performs advanced monitoring of rainfall over the tropics/subtropics

„ Instruments Precipitation Radar (PR) Visible and Infrared Scanner (VIRS) TRMM Microwave Imager (TMI) Clouds and Earth’s Radiant Energy System (CERES) Lightning Imager Sensor (LIS)

„ Conducts 18 real-time supports per day through TDRSS

„ Extensive on-board automation and fault management for self-safing

„ Robust telemetry stream provides ample fault status messaging

GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 3 TRMM Re-Engineering Objectives

„ Reduce overall spacecraft and data ops costs by 50% in FY 2004

Feb 2004: Automate to reduce console engineers from 2 to 1 per shift

Oct 2004: Enable lights out operations for the midnight shift

Nov 2004: Reduce H/W maint. costs by eliminating outdated equipment

„ Implement, deploy and begin use of the new system without impacting ongoing operations

„ Demonstrate the GMSEC architecture as a proof of concept to use as the basis for automating other NASA missions (Terra, , Aura)

„ Deploy an operational system in 14 months with less than $2M

GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 4 Technical Approach „ Employ a phased approach to meet immediate and long term budgetary concerns „ Add new tools to old system to enable immediate staff reduction (problem recognition and user paging) „ Replace front-end processor Conversion from Serial to Internet Protocol (IP) eliminates outdated H/W „ Replace legacy telemetry and command system Will now use Raytheon’s Eclipse instead of in-house system Standardizes Earth Science T&C systems to reduce H/W and S/W maintenance costs „ Integrate ground components into GMSEC architecture to allow use of automation and monitoring tools „ Introduce simple ground automation Routine pass activities String failovers „ Allow for future expanded use of automation tools

GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 5 Automation Philosophies

„ Current NASA Goddard missions use custom automation software GOTS Expert Systems are composed of fully integrated automation components, such as paging, automatic scheduling, analysis, fault isolation and recovery Customized Scripting Tools developed in-house for mission unique automation „ TRMM will provide user configurable automation toolset Contemporary COTS provide more automation eliminating the need for complex, integrated automation tools This approach facilitates “plug-and-play” components to support evolving technology „ Standard interface enables an infrastructure that is independent of its components

GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 6 TRMM Automation Approach

„ Implement two automation approaches: event- driven and time-driven Event-driven based on automatic responses to ground generated event using rules/actions Sends directive to paging sys to text page engineers when spacecraft limits violated and critical ground events occur Sends directive to trigger automatic ground reconfigurations Time-driven based on automatic execution of pass activities through static and dynamic pass plans Automate spacecraft real-time activities during Configure ground system pre-pass, real-time support, and post-pass

GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 7 GMSEC Architecture Integration

„ Development and test approach Components to modify their software to support standard interface and message formats Establish Control Center test facility to test ground component communication on the software bus „ Middleware software bus TIBCO Smart Sockets offers guaranteed delivery of event messages and directives FY 2004: Implement event messages, directives, and heartbeat messages FY 2005: Integrate further ground system automation, as well as transfer telemetry and products via the bus

GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 8 GMSEC System Architecture

DSMC SN/GN

l Simulator a Formats FDF PACOR

UPS m

c

l

i

T

t C /

s

i

m s

e

l r P

d

d i P l s

a F

/ m m

C ro n T l

l 0

C T d n m i r l n u n e c g d v

L o t r

i s o c

s Commands e R s Planning/ Telemetry & Front End Flight Dynamics Trending Mi Scheduling (AutoFDS) Command Sys Processor (GTAS) (MOPSS) (Eclipse) (IRTS)

s Telemetry s

s s

e

e

s

e e

t

s

s v

v

v v

s

t

i

i

t

n i i

t

t

t

t t

n

n

e

c

n c

c c

e

e

v

e

e e

e e v

v

r

r

E v

r r

i

i

i i

E

E

E

D

D

D D GMSEC Infrastructure (Messaging Service)

s

s

s

t

e

s

t

s

e

s

v

t n

e a

i

s

v

e

e

t v n

e

i t

i

t

v

v

c e

t

b

n

i

n

c t

t

v e

E

c

e

r

r

e

c

e

i E

v

o r a

r

e i

i

i

E e

D

r

t

i

D

D

H

D

a

m Automated Criteria Action Table Auto

o t Pass Scripts Criteria Action Notification Event Archive

u (Scenario Scheduler) •Ground event •Issue directive (ANSR)

A messages •Issue event msg

s

s

n

s

a

a

l

P

P FTP from Planning Sys GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 9 Final Architectural Diagram

Real Time T&C Subnet

Automation Subnet

Existing System

GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 10 Challenges

„ Major challenge has been to implement an operational system in 14 months without exceeding budget

„ Extensive Control Center changes requires a continuous re-assessment of risk

„ Re-engineering an on-orbit ground system adds complexity to testing and rehosting new systems

„ Critical support team was difficult to capture due to transition of mission support contract

„ Lack of documentation causes code reverse engineering, driving costs higher than anticipated

GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 11 Conclusions

„ The TRMM re-architecting effort offers ground- breaking technological advances for NASA missions

„ Pioneering this innovative approach will enable its reuse for other NASA missions

„ Current NASA Goddard mission automation paradigm will evolve to a message-oriented infrastructure with simple automation tools

GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 12