Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center
Ground System Architectures Workshops (GSAW2004) Manhattan Beach, California March 30, 2004
Lori Enright, NASA/Goddard Space Flight Center Charlie Bengston, Ronald A. Jones/ASRC Aerospace Project Background
Budgets for NASA space missions typically extend through the end of their original design lifetime
Scientifically valuable missions which keep operating into an “extended mission phase” are faced with severe operating cost limitations
TRMM was launched in 1997 with a planned operational life of 3 years - our desire is to extend the mission through 2011
For TRMM, the NASA Earth Science Mission Operations (ESMO) Office is creating a “pathfinder” system based on GSFC’s plug-and- play message-oriented GMSEC architecture
GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 2 NASA’S Tropical Rainfall Measuring Mission
Launched in Nov 1997 with a design lifetime of 3 years
Low earth orbit: 402 km, 35 degree inclination
Performs advanced monitoring of rainfall over the tropics/subtropics
Instruments Precipitation Radar (PR) Visible and Infrared Scanner (VIRS) TRMM Microwave Imager (TMI) Clouds and Earth’s Radiant Energy System (CERES) Lightning Imager Sensor (LIS)
Conducts 18 real-time supports per day through TDRSS
Extensive on-board automation and fault management for self-safing
Robust telemetry stream provides ample fault status messaging
GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 3 TRMM Re-Engineering Objectives
Reduce overall spacecraft and data ops costs by 50% in FY 2004
Feb 2004: Automate to reduce console engineers from 2 to 1 per shift
Oct 2004: Enable lights out operations for the midnight shift
Nov 2004: Reduce H/W maint. costs by eliminating outdated equipment
Implement, deploy and begin use of the new system without impacting ongoing operations
Demonstrate the GMSEC architecture as a proof of concept to use as the basis for automating other NASA missions (Terra, Aqua, Aura)
Deploy an operational system in 14 months with less than $2M
GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 4 Technical Approach Employ a phased approach to meet immediate and long term budgetary concerns Add new tools to old system to enable immediate staff reduction (problem recognition and user paging) Replace front-end processor Conversion from Serial to Internet Protocol (IP) eliminates outdated H/W Replace legacy telemetry and command system Will now use Raytheon’s Eclipse instead of in-house system Standardizes Earth Science T&C systems to reduce H/W and S/W maintenance costs Integrate ground components into GMSEC architecture to allow use of automation and monitoring tools Introduce simple ground automation Routine pass activities String failovers Allow for future expanded use of automation tools
GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 5 Automation Philosophies
Current NASA Goddard missions use custom automation software GOTS Expert Systems are composed of fully integrated automation components, such as paging, automatic scheduling, analysis, fault isolation and recovery Customized Scripting Tools developed in-house for mission unique automation TRMM will provide user configurable automation toolset Contemporary COTS provide more automation eliminating the need for complex, integrated automation tools This approach facilitates “plug-and-play” components to support evolving technology Standard interface enables an infrastructure that is independent of its components
GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 6 TRMM Automation Approach
Implement two automation approaches: event- driven and time-driven Event-driven based on automatic responses to ground generated event messages using rules/actions Sends directive to paging sys to text page engineers when spacecraft limits violated and critical ground events occur Sends directive to trigger automatic ground reconfigurations Time-driven based on automatic execution of pass activities through static and dynamic pass plans Automate spacecraft real-time activities during night shift Configure ground system pre-pass, real-time support, and post-pass
GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 7 GMSEC Architecture Integration
Development and test approach Components to modify their software to support standard interface and message formats Establish Control Center test facility to test ground component communication on the software bus Middleware software bus TIBCO Smart Sockets offers guaranteed delivery of event messages and directives FY 2004: Implement event messages, directives, and heartbeat messages FY 2005: Integrate further ground system automation, as well as transfer telemetry and products via the bus
GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 8 GMSEC System Architecture
DSMC SN/GN
l Simulator a Formats FDF PACOR
UPS m
c
l
i
T
t C /
s
i
m s
e
l r P
d
d i P l s
a F
/ m m
C ro n T l
l 0
C T d n m i r l n u n e c g d v
L o t r
i s o c
s Commands e R s Planning/ Telemetry & Front End Flight Dynamics Trending Mi Scheduling (AutoFDS) Command Sys Processor (GTAS) (MOPSS) (Eclipse) (IRTS)
s Telemetry s
s s
e
e
s
e e
t
s
s v
v
v v
s
t
i
i
t
n i i
t
t
t
t t
n
n
e
c
n c
c c
e
e
v
e
e e
e e v
v
r
r
E v
r r
i
i
i i
E
E
E
D
D
D D GMSEC Infrastructure (Messaging Service)
s
s
s
t
e
s
t
s
e
s
v
t n
e a
i
s
v
e
e
t v n
e
i t
i
t
v
v
c e
t
b
n
i
n
c t
t
v e
E
c
e
r
r
e
c
e
i E
v
o r a
r
e i
i
i
E e
D
r
t
i
D
D
H
D
a
m Automated Criteria Action Table Auto
o t Pass Scripts Criteria Action Notification Event Archive
u (Scenario Scheduler) •Ground event •Issue directive (ANSR)
A messages •Issue event msg
s
s
n
s
a
a
l
P
P FTP from Planning Sys GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 9 Final Architectural Diagram
Real Time T&C Subnet
Automation Subnet
Existing System
GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 10 Challenges
Major challenge has been to implement an operational system in 14 months without exceeding budget
Extensive Control Center changes requires a continuous re-assessment of risk
Re-engineering an on-orbit ground system adds complexity to testing and rehosting new systems
Critical support team was difficult to capture due to transition of mission support contract
Lack of documentation causes code reverse engineering, driving costs higher than anticipated
GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 11 Conclusions
The TRMM re-architecting effort offers ground- breaking technological advances for NASA missions
Pioneering this innovative approach will enable its reuse for other NASA missions
Current NASA Goddard mission automation paradigm will evolve to a message-oriented infrastructure with simple automation tools
GSAW 2004 – March 30, 2004 Low-Cost Re-Architecting of NASA’s TRMM Mission Control Center 12