cluster-II operations concept

Cluster-II: Evolution of the Operations Concept

M. Warhaut, S. Matussi & P. Ferri Mission Operations Department, ESA Directorate of Technical and Operational Support, ESOC, Darmstadt, Germany

Introduction associated with possible or enforced changes The preparation of this recovery action for a in several hardware components. The problem major space science mission, to be launched was complicated by the fact that one of the four years after the original one, represents a four spacecraft was already completely significant engineering and managerial integrated based on spare parts from the original challenge. The need to cap the new mission at Cluster programme. This spacecraft, called less than 50% of its original cost has imposed ‘Phoenix’ or Cluster-FM5, is identical to the a rigid discipline in terms of keeping the original spacecraft, but slightly different from spacecraft-component and ground-segment the next three to be built. The not completely facility configurations as close as possible to identical spacecraft hardware has meant those used for the original mission. On the reduced flexibility during the integration and other hand, some changes have had to be test phases. In the original Cluster programme, accepted as inevitable, due for example to a spacecraft units were often exchanged from lack of spare parts for the spacecraft, or to one spacecraft to another depending on keep pace with the continuous evolution in the hardware availability and the need to continue ESOC ground segment. specific test activities. This flexibility was one of the keys to the success of the original Cluster The operations concept for the Cluster-II mission has had to evolve integration programme, which ensured the with respect to the original Cluster baseline, due mainly to changes in continuation of the complex activities without the spacecraft design and the reduction in the number of ground accumulating significant delays. In addition, it stations from two to only one for routine mission-operations support. had always been a fundamental assumption The solutions adopted have allowed the overall impact on the ground underlying the design of the ground segment segment and mission operations to be minimised, whilst still and the operations concept that all four maintaining the scientific data return at the original level. spacecraft would be indeed identical. The non- availability of original parts already utilised on the Phoenix spacecraft has meant that Cluster-II also suffers from the fact that it will be exceptions have had to be accepted. launched during what was already an extremely busy period for ESOC, with the launch and The main changes in spacecraft hardware due control of ESA’s XMM mission, and the to unavailability of now obsolete parts are in the provision of launch support for other external telecommunications (TTC) and on-board data- missions (e.g. Meteosat Second Generation). handling (OBDH) subsystem areas. The original This results in the need to share the existing TTC high-power amplifier was no longer available facilities, in particular the ground stations. and had to be replaced on the three new Cluster-II has therefore had to accept to use a spacecraft, together with the transponder, by single dedicated ground station for the new hardware derived from that developed for mission’s routine science operations phase, ESA’s XMM and Integral scientific satellites. A instead of the two originally foreseen, which major OBDH change is replacement of the two has had a significant impact on the operations original solid-state recorders (SSRs) with a concept. An additional consequence of the single recorder of a new design with a higher ‘year 2000 peak’ is the difficulty in re-utilising recording capacity. No changes in the operating staff with Cluster experience, most of whom are philosophy of the TTC subsystem are required, already supporting other missions. but the operational database and the related flight-control procedures have been affected. Space-segment evolution On the other hand, the new SSR allows the In re-building the new Cluster spacecraft, there way data recording and dumping is managed were from the outset various dilemmas by the ground to be significantly improved.

61 r bulletin 102 — may 2000 bull

As far as the payload is concerned, there will be For the Cluster-II programme, four separate an identical complement of instruments flying SVT slots were allocated, each with a single, on the four spacecraft, and so at least the different flight model and for a maximum instrument hardware will be identical to the duration of four days. original set. The on-board software for several instruments has changed, however, and This limited test time imposed the need for a possible impacts on the ground segment have careful trade-off in the selection of subsystems been carefully analysed, and also checked in and functions to be addressed, in order to the test and verification phase. concentrate mainly on those areas in which changes with respect to the original spacecraft It was decided, for cost and schedule reasons, were to be expected. This approach relied on not to change the on-board software of the the correctness and completeness of the OBDH central on-board processor, although documentation describing the changes, and several patches to improve the final software therefore bore inherent risks. These risks had, version were already prepared prior to the however, to be accepted due to the tight original Cluster launch. For Cluster-II, these project schedule, and were kept within patches will be loaded on top of the software reasonable limits since the overall number of already burned into the spacecraft PROMs changes introduced into the spacecraft has before the launch. For the ground segment, this been small and strictly controlled (Table 1). means safer operations immediately after launch. Ground-segment evolution Unlike the problems encountered in re-building The changes in integration approach from the the space segment linked mainly to original Cluster programme, introduced to unavailability of parts, the ground segment has speed up the production work and taking into had to deal with a continuously evolving account the already accumulated experience, infrastructure. This evolution could not be had an impact on the testing approach for the halted for four years to wait for the re-launch of ground segment. The traditional final system the Cluster mission and then support it with the test for ESOC, the System Validation Test same systems. Apart from the modernisation of (SVT), in which the ground segment exercises the infrastructure, which is a continuous process and verifies all command and telemetry and normally only marginally constrained by the functions with the spacecraft flight model, was needs of the projects using it, one of the originally carried out with two spacecraft in problems faced today is rapid obsolescence of parallel. This was done to validate one of the computer hardware and software. Workstations, basic features of the Cluster operations, namely operating systems and, in general, commercial the parallelism of control activities on more than off-the-shelf (COTS) products have an average one spacecraft. This approach also had the lifetime of just 18 months, after which the benefit of increasing the test time available to maintenance costs – if maintenance is even ESOC with the flight hardware. In the original supported by the supplier – become far larger programme, each spacecraft was tested from than the cost of replacing the item with the ESOC for more than 15 working days in total. latest model or version. This carries with it,

Table 1. Summary of changes to the space segment

Change Reason Impact

New Transponder/High Power Amplifier Old High Power Amplifier not procurable Different procedures and databases in the TTC for three spacecraft area between Phoenix and the rest of the fleet. Upgrade of ESOC software simulator needed

New Solid-State Recorder (SSR) Old Solid-State Recorder (SSR) not Positive impact on operations, since the new SSR procurable. New SSR has higher capacity allows partial dumps. Upgrade of ESOC software and greater flexibility of use simulator needed

Patches to on-board software burned Changes in software were necessary, but Safer LEOP ops compared to original flight (patches into PROM re-build of full software considered too to be loaded at launch site). However, no expensive “clean” starting point for software maintenance

New payload software Evolution of scientific knowledge and Database/procedures changes necessary. targets Heavy SVT re-testing. Minor upgrade of ESOC software simulator

Sequential spacecraft integration Acceleration of production schedule Parallel SVT on two spacecraft not possible. and testing Limited testing time for new features

62 cluster-II operations concept

however, the problem of adapting, i.e. ‘porting’, same time frame, which all utilise packet the application software to the new tools or telemetry and telecommand standards. In the platforms, with an inherent, non-negligible cost telemetry area, the solution adopted was to and schedule impact, and with the related risk port the Cluster telemetry processor software of not meeting the original specifications. to the new Solaris 2.6 operating system.

The Cluster ground segment suffered this The new operating system will allow both the problem in all critical areas: ground stations, old telemetry processor software (TMP3) and control system and simulators. A main the new software (TMP4) to be run on the same component of the on-going upgrading process platform, with the required performance. In is the porting of the software to new operating addition, the change to a new hardware systems: from Sun OS to Solaris 2.6 for the platform (UltraSparc workstations), which very ground-station equipment based on Sun soon became mandatory (maintenance costs workstations, and to higher VAX VMS versions for the old Sparc20 platforms were becoming or Alpha stations for the control system and the prohibitive) did not imply any additional simulator. The infrastructure changes dictated software adaptation exercise. This solution by software and hardware obsolescence allowed the installation of identical hardware on created significant problems in the area of all ESA workstations. In order to support ground-station interfaces in particular. different missions, a simple restart of the Compromise solutions, mixing the old and the telemetry processor using different software new interfaces, have been adopted to minimise (TMP3 or TMP4) is required. The telecommand changes to the old baseline. Figure 1 shows interface to the Control Centre has also the new baseline for all interfaces between the changed, but fortunately the solution adopted ground stations and the mission-control will allow the utilisation of the new system managed by a central computer, the telecommand encoder software and hardware Network Control and Telemetry Routing also for the Cluster-II mission, via normal System (NCTRS), located at the Control configuration changes. Centre. Another change imposed on the ground Ground stations are one of the main stations, this time due to hardware infrastructure items of the ground segments for obsolescence, was the development of a new the missions supported by ESOC. They are Station Computer (STC), the central local shared by all missions, particularly for the control system for all ground-station units. Its launch and early orbit phases. For these repercussions for Cluster-II lie mainly in the area important shared items, it is essential that of the interface to the Mission Planning System identical, or at least compatible, interfaces to (MPS), which produces schedules to be the mission control systems are used. On the transferred to the station computer for other hand, the Cluster requirements on automatic station control. The scheme adopted telemetry and telecommand interfaces to the in the MPS software for the generation of the ground stations are different from those of the STC schedules is incompatible with the way other missions that will be supported in the schedules are handled in the new station

Quick Look Station Command Telemetry Tracking NASA TM and Encoder Computer Processor System Ranging Back-up TC STC2 TCE TMP 3 (Solaris 2.5) MK IV & MK II emulation Session Layer ( layer 5) Session Layer ( layer 5) STDM X.25 Schedules X.25 Protocol CMIS/CMIP CMIS/CMIP (application Layer) (application Layer)

STC2 Network Control Remote and Telemetry Workstation Routing System NCTRS VAX Open VMS 7.1

STDM Tracking Schedule contents (ascii) TCP/IP No change TCP/IP FTP(ascii) FTP

New (agreed) Flight Data Dedicated Mission Figure 1. Network Control New (to be solved) Dynamic Disposition Computer Planning and Telemetry Routing System System System System System (NCTRS) interfaces

63 r bulletin 102 — may 2000 bull

computer. The adaptation of this interface most significant modifications has been the involved work on both the STC and the MPS replacement of the servo and tracking systems, software. The changes in the interfaces necessary because the Cluster-II satellites will between the Control Centre and the ground move in a highly elliptical orbit and require high- stations with respect to the original mission are speed tracking. About 0.8 Gbyte of data will be shown in Figure 1. returned each day from the 44 experiments (11 scientific instruments on each of the four The fact that one of the two original dedicated spacecraft). Over two years of operations, this ground stations is no longer available triggered adds up to 580 Gbyte (580 000 000 000 byte) a major change in the Cluster-II baseline, which of data – equivalent to 290 million pages of now has only one ground station – at printed text. All of the Cluster-II data exchange Villafranca, near Madrid (E) – to support the between Villafranca and ESOC will be via routine science mission phases. A single dedicated communications lines. antenna will therefore be used to control the four Cluster spacecraft sequentially. This Another ‘victim’ of hardware and software change prompted a number of studies and obsolescence is the Cluster software simulator. trade-off activities, resulting in an operations This software tool is based on two computers, concept that provides a significant reduction in a DEC Alpha workstation to simulate the four operating costs, in terms of both manpower spacecraft, and a DEC VAX workstation to run and facilities. the ground-segment models. This separation was needed because of the high computer The main antenna to be used at Villafranca, processing load when simulating four known as VIL-1, is a 15 m dish that operates in independent spacecraft. The operating system S-band at 1.8 – 2.7 GHz (Fig. 2). Formerly used of the VAX used to simulate the ground stations for the IUE and ISO missions, its hardware and and communication network is no longer electronic equipment have recently been maintained by the manufacturer and needed to refurbished and upgraded to comply with be upgraded. For this and other reasons, it was Cluster-II requirements. Modernisation of VIL-1 decided to port the Cluster ground models also involved transporting more than 23 t of to the new Alpha workstations, upgrading to equipment from the Odenwald site in Germany, the latest VMS operating system, rather than which was a two-week-long road journey. maintain the obsolete software. Furthermore, Since its arrival at Villafranca in November the Cluster simulator had to be updated to 1998, much of the hardware has been follow the spacecraft design modifications replaced, including the 60 dish panels, the and ensure that it is functionally representative Figure 2. The Cluster-II antenna at ESA’s Villafranca subreflector, the antenna equipment room and of the behaviour of the new Cluster-II station, near Madrid (E) other parts of the main structure. One of the spacecraft. The impact of the spacecraft changes on the software simulator was confined to the transponder and SSR. Changes in the payload had only a small overall effect on the simulator.

The Control Centre facilities at ESOC (Fig. 3) have also been affected in that the original Cluster Dedicated Control Room (DCR) now has to be shared with the XMM project, which is already using it. The original room was designed for a double controller position, each in charge of two spacecraft and with parallel operations via the two Cluster dedicated ground stations. The room included eight identical spacecraft control work- stations and two station control workstations. Thanks to the use of a single ground-station antenna for Cluster-II, it will be possible to use a single station computer workstation in the DCR. Also, fewer spacecraft control workstations will be available. This constraint will be acceptable because only one spacecraft will be in

64 cluster-II operations concept

Figure 3. The Main Control Room at ESOC

visibility of the ground segment at any given stack of two Cluster spacecraft. There is a four- moment, allowing a single spacecraft controller week interval between the two launches. to carry out all the necessary real-time operations. In this case, no more than four Because of the difference in latitude between workstations will be needed for spacecraft- Baikonur (52 deg) and Kourou (5 deg), the control activities. timelines for the Launch and Early Orbit Phase (LEOP) and the Transfer Orbit Phase (TOP) to The ground-segment changes are summarised the final operational orbit look very different in Table 2. from those for the original Cluster mission. For the Cluster-II initial operations phase, the ESA Mission operations concept evolution ground stations of Kourou (Fr. Guiana), Perth The tight budgetary constraints on the mission (W. Aus.), Kiruna (Sweden) and Villafranca have imposed many changes on the Cluster-II (Spain), plus the DSN Canberra (Aus.) station operations scenarios, including the launches by will be available. The LEOP/TOP operations Russian Soyuz-type vehicles from Baikonur timeline will be defined such that critical (Kazakhstan). The launch scenario foresees activities (such as orbit manoeuvres) on the two two Soyuz launchers, enhanced with a spacecraft are not executed in parallel. This dedicated fourth upper stage, each carrying a reduces the size of the mission operations

Table 2. Summary of ground-segment changes

Change Reason Impact

MCS VAX software ported to Alpha Lack of maintenance support for old versions None expected

TMP software ported to Solaris 2.6 Compatibility of TMP hardware platform with None expected operating system other missions

New station computer (STC 2) STC-1 hardware obsolescence Changes in mission-planning software to adapt to new STC schedule handling

Software simulator porting from VAX Lack of maintenance support for old versions None expected VMS to Alpha

Single ground-station support for routine Heavy ESOC workload imposes sharing of ground Operations concept modified; science ops. phase station with other missions; cost reduction upgrade of mission planning software required; ops. manpower reduction.

Reduction of floor space in Dedicated Heavy ESOC workload imposes sharing of OCC Only possible thanks to revised ops. concept Control Room facilities with other missions (sequential and not parallel spacecraft control)

65 r bulletin 102 — may 2000 bull

team needed and allows a better distribution of As already mentioned, during the Mission the available expertise across the two shifts. Operations Phase (MOP) in which the scientific Once the first pair of spacecraft have been put observations will be performed, only one into their final operational orbit, the second pair ground station (Villafranca) will be used for will be launched and a second LEOP/TOP science data recovery, and the baseline is to phase will begin. The operations related to the use a single antenna to serve all four spacecraft. second launch will be complicated by the This is a major change in the operations presence of the first pair of spacecraft, which scenario compared with the original mission, will need to be monitored and controlled from which used two ground stations, each one time to time by the same operations team. permanently dedicated to two spacecraft. It implies that, on average, each spacecraft is Figure 4 is a schematic of the LEOP/TOP visible from the ground station for only half of operations timeline, and how these critical the time that was previously available. Studies phases for the two launches are connected. L1 have been performed to analyse how much is the time of launch of the first spacecraft pair, science data can be recovered with this new L2 the second launch, nominally four weeks configuration, and what on-board storage is later. required for the new Solid State Recorder (SSR) in order to compensate for the reduced The deployment of spacecraft appendages spacecraft visibility. The results show that it will (instrument and lower-antenna booms) and the still be possible to recover the same quantity of start of payload-commissioning activities for all science data that was specified in the original four spacecraft only takes place once the full Cluster Master Science Plan (MSP). However, constellation has been achieved, i.e. all some changes must be implemented in the spacecraft are in the initial operational orbit. ground segment to cope with the reduced The reduced ground-station availability has visibility periods, such as the doubling of the imposed a major change in the Commissioning data-link capacity between the station and the and Verification Phase (CVP) operations. Control Centre, and the possibility to execute Originally these operations were to be executed partial dumps of the SSR stored telemetry data. in parallel for two spacecraft, using the two With the latter possibility, it is feasible to exploit dedicated ESA ground stations. For Cluster-II, every single visibility slot, thereby maximising the single mission-dedicated station in the science data return. Partial dumps can only Villafranca (E) will be used, augmented by the be performed in forward mode, i.e. older data DSN Canberra station. As the ground coverage first, to avoid the need for reconstituting the of the two stations is almost complementary, temporal sequence of science and house- CVP operations for the four spacecraft will be keeping data. This is also a change from the executed sequentially, but covering almost 24 original Cluster approach, which was to dump hours of real-time activities per day. The new all data from the SSR in reverse order, involving mission plan for this phase is still being a modification of the ground software that finalised, but it is expected that all activities can processes the dumped data. be covered in a time comparable to that assumed for the original mission (12 to 14 An advantage of using a single ground station weeks). The current baseline is to start CVP controlling the four spacecraft in sequence is operations on the first pair of spacecraft only that routine mission operations can be after the second pair have completed their executed by a single spacecraft controller LEOP/TOP activities. position, compared to the two of the original Figure 4. The LEOP/TOP timeline mission, significantly reducing costs. Defining the size and timing the recruitment of the mission control team for a recovery mission is Delta Vs for Apogee raising and Inclination change / Perigee raising always difficult. The danger is to underestimate the unavoidable changes required in the Orbit trims and operational documentation, such as the flight Initialise System to support second S/C pair control procedures, and the modifications to the control system, and therefore implement a Delta Vs for Apogee raising and late build-up of a, perhaps undersized, flight Inclination change / Perigee raising control team. The case of the Cluster-II mission is complicated by the fact that only 3 of the Orbit trims and Boom deployment; original 23 members of the Cluster control team start of Commissioning will participate in the new mission. The problem of maintaining the expertise and skills that were L1 L2 available in the original team is partly mitigated by the fact that many of the initial team Week 1 Week 2 Week 3 Week 4 members are still available at ESOC, having

66 cluster-II operations concept

moved to other projects. Part-time involvement and functionality and to an improved mission of some experts from the original Cluster control concept, significant cost reductions in mission is therefore already a reality, and will terms of manpower requirements and facilities continue until the critical phases of the mission utilisation have been achieved without are over. impacting the overall science data return.

Conclusion Acknowledgements The preparation of a ground segment and The work described in this article was carried mission-operations concept for the Cluster-II out on all sides – science, industry, project and recovery mission, heavily affected by severe operations – by small teams of ‘veterans’ of the cost and schedule constraints, was driven by original Cluster mission, using their reservoir of the basic premise of trying to avoid any change skills and knowledge accumulated over several to the original baseline. At the same time, the years of being dedicated to this project. The unavoidable ‘environmental’ changes, such as realisation of a recovery mission within the the replacement of obsolete parts in the limited resources assigned to the Cluster-II spacecraft or the adaptation to the new project is only possible thanks to the work of ground-segment infrastructure items, have had many more people – the hundreds of scientists, to be taken into account. technicians and engineers who originally developed, integrated and tested the Cluster In some cases, changes were imposed purely spacecraft, payload and ground segment, and by the need for cost savings, including the who left the project for many different reasons change to a single ground station for controlling after the failed launch. To these people in the routine phases of the mission. Thanks to the particular the authors are sincerely grateful. r upgrades to spacecraft data-storage capacity

1/2 page ADV

67