The AGILE Data Center and Its Legacy

Noname manuscript No. (will be inserted by the editor)

The AGILE Data Center and its Legacy

Carlotta Pittori, on behalf of the AGILE-SSDC Team

Received: date / Accepted: date

Abstract We present an overview of the main AGILE Data Center activities and architecture. AGILE is a space mission of the Italian Space Agency (ASI) in joint collaboration with INAF, INFN, CIFS, and with the participation of several Italian space industry companies. The AGILE satellite was launched on April 23, 2007, and is devoted to the observation of the gamma-ray Uni- verse in the 30 MeV – 50 GeV energy range, with simultaneous X-ray imaging capability in the 18–60 keV band. The AGILE Data Center, part of the ASI multi-mission Space Science Data Center (SSDC, previously known as ASDC) is in charge of all the scientiﬁc operations: data management, archiving, distribution of AGILE data and scientiﬁc software, and user support. Thanks to its sky monitoring capability and fast ground segment alert system, AGILE is substantially improving our knowledge of the gamma-ray sky, and provides a crucial contribution to multimessenger follow-up of gravitational waves and neutrinos. Keywords gamma rays: observations; astronomical data bases: miscella- neous; data center

1 Introduction

AGILE (Astrorivelatore Gamma ad Immagini LEggero) is an Italian scientific space mission for high energy astrophysics funded by the Italian Space Agency (ASI) with scientific and programmatic participation by INAF, INFN, CIFS, several Italian universities and industrial contractors. The AGILE very innovative instrument combines for the first time a gamma-ray imager based

C. Pittori ASI-SSDC, via del Politecnico, I-00133 Roma, Italy INAF-OAR, Via Frascati 33, I-00078 Monte Porzio Catone (RM), Italy E-mail: [email protected] ORCID iD: 0000-0001-6661-9779 arXiv:1911.12314v1 [astro-ph.IM] 27 Nov 2019 2 Carlotta Pittori, on behalf of the AGILE-SSDC Team

Fig. 1 The AGILE payload. on solid-state silicon technology, and a hard X-ray imager. The AGILE satellite was successfully launched on 2007 April 23rd from the Indian base of Sriharikota, and it was inserted in an equatorial low Earth orbit with an in- clination of 2.5 degrees and average altitude of 535 km. The AGILE instrument shown in Fig. 1 is a cube of 60 cm side, weighting only about 100 kg. It consists of two detectors using silicon technology: the gamma-ray imager GRID and the hard X-ray detector SuperAGILE for the simultaneous detection and imaging of photons in the 30 MeV - 50 GeV and in the 18 - 60 keV energy ranges. The payload is completed by two non-imaging detectors: a Mini-Calorimeter (MCAL) sensitive in the energy range 350 keV to 100 MeV, and an anti-coincidence system. Further details on the AGILE scientific mission are reported in [1] and Tavani (2019), these Proceedings. The AGILE satellite, designed to achieve a nominal lifetime of two years, has completed more than ten years of operations in orbit, and it is substantially contributing to improve our knowledge of the high-energy sky, also providing an important contribution to multimessenger follow-up of gravitational waves and neutrinos. Satellite operations and all payload functions are nominal and AGILE continues its mission with high efficiency. In this paper we describe the architecture and functionalities of the AGI- LE Data Center, which is part of the ASI multi-mission Space Science Data Center (SSDC, previously known as ASDC). In Sect. 2 we present the general architecture of the AGILE Data Center, and in Sect. 3 the AGILE data flow. We describe in Sect. 4 the AGILE observation modes, in Sect. 5 the data processing system and data levels, in Sect. 6 the “AGILE Services” and the AGILE storage, in Sect. 7 the data policy and distribution, and in Sect. 8 the AGILE published catalogs and their interactive webpages. Finally in Sect. 9 we present the AGILE legacy archive and the AGILE-LV3 web tool for scientific analysis. The AGILE Data Center and its Legacy 3

Fig. 2 The Ground Segment (GS) architecture of the AGILE space mission. The AGILE Data Center acts both as a Science Operation Center (SOC) and as a Science Data Center (SDC).

2 The AGILE Data Center

The AGILE Data Center (ADC)1 is in charge of all the scientific oriented activities related to the analysis, archiving and distribution of AGILE data. It is part of the ASI Space Science Data Center (SSDC) located at the ASI Headquarters in Rome (Italy), and it includes scientific personnel from both the SSDC and the AGILE Team, with the support of the IT technical Team of the multi-mission ASI data center. In the context of the Ground Segment (GS) architecture of the mission, the AGILE Data Center acts both as a Science Operation Center (SOC) and a Science Data Center (SDC), see scheme in Fig. 2. The main ADC activities and responsibilities include: scientific requirements implementation, development and implementation of data processing systems (pipelines), relational databases and data archives. Management of the satellite observations include: real time data acquisition and monitoring, alert systems, processing (and reprocessing), archiving, standardization, analysis and interpretation of data, software tools development to grant optimal diffusion, accessibility and usability of scientific data. ADC also managed the Guest Observer (GO) Program of the AGILE mission, and it is responsible for the standardization, inclusion and distribution of public scientific data in the large SSDC interactive multi-mission archive. More details are given in the following sections.

1 https://agile.ssdc.asi.it/ 4 Carlotta Pittori, on behalf of the AGILE-SSDC Team

Fig. 3 Pictorial view of the data ﬂow of the AGILE Ground Segment.

3 The AGILE Data Flow

AGILE raw telemetry level-0 data (LV0) are down-linked every ∼ 100 min to the ASI Malindi ground station in Kenya and transmitted, through the fast ASINET network provided by ASI, ﬁrst to the Telespazio Mission Control Center at Fucino, and then to the ADC within ∼ 5 min after the end of each contact downlink, see Fig. 3.

The AGILE-GRID ground segment alert system is distributed among ADC and the AGILE Team Institutes and it combines the ADC quick-look with the AGILE Science Alert System developed by the AGILE Team [2]. GRID Alerts are sent via email (and sms) both on a contact-by-contact basis and on a daily timescale. The already very fast AGILE ground segment alert system (with alerts within ∼ (2-2.5) hours since an astrophysical event) has been further optimized in May 2016, for the search of electromagnetic counterparts of gravitational waves and other transients. Currently the system allows the AGILE Team to perform a full data reduction and the preliminary Quick Look (QL) scientiﬁc analysis only 25/30 minutes after the telemetry download from the spacecraft.

Reﬁned analysis with human intervention of most interesting alerts are performed every day (quick-look daily monitoring). The AGILE Data Center and its Legacy 5

Fig. 4 The large AGILE-GRID FoV in spinning on a particular day. The sky regions not accessible for solar panel constraints are delimited in red, and indicate the mean daily sun and anti-sun positions.

4 AGILE observation modes

4.1 Pointing mode

The AGILE pointings are subject to illumination constraints requiring that the fixed solar panels always be oriented within about 3 degrees from the Sun direction. During the first ∼ 2.5 years AGILE was operated in “pointing observing mode”, characterized by long observations called Observation Blocks (OBs), typically of 2-4 weeks duration, mostly concentrated along the Galactic plane following a predefined Baseline Pointing Plan. The AGILE Pointing Plan during the first two years of observations (Cycle-1 and Cycle-2) has been prepared taking into account several scientific and operational requirements such as: maximisation of the overall sky exposure by limiting the observation of the sky regions more affected by Earth occultation; substantial exposure of the Galactic plane, in particular of the Galactic Center and of the Cygnus region, in order to achieve good statistic for several gamma-ray pulsar candidates, micro-quasars, supernova remnants, and unidentified gamma-ray sources with simultaneous hard X-ray and gamma-ray data. During the pointing period phase, even though the exposure was mainly focused towards the Galactic plane, the large AGILE FoV has allowed to achieve a good balance between Galactic and extra-galactic targets, with fast reaction capability to transient 6 Carlotta Pittori, on behalf of the AGILE-SSDC Team

Fig. 5 AGILE-GRID total intensity map for energies above 100 MeV (Pointing + Spinning) from December 1, 2007 up to Sep. 30, 2017. gamma-ray sources. The list and details of the 101 OB of AGILE observations in pointing mode can be found at the ADC dedicated webpages2.

4.2 Spinning mode

On November 4, 2009, AGILE scientific operations were reconfigured following a malfunction of the rotation wheel occurred in mid October, 2009. Since then, the satellite is operating regularly in “spinning observing mode”, with the solar panels pointing at the Sun and the instrument axis sweeping the sky with an angular speed of about 0.8 degree/sec. The instrument and all the detectors are operating nominally producing data with quality equivalent to that obtained in pointing mode. AGILE in spinning mode is surveying a large fraction (about 80%) of the sky each day. The AGILE Guest Observer Program has not suffered any interruption. An example of the accessible sky view in spinning on a particular day is shown in Fig. 4 and the total 10-year AGILE intensity map above 100 MeV from December 1, 2007 up to September 30, 2017 (Pointing + Spinning) is shown in Fig. 5. AGILE observations in spinning continue to be structured as a series of OBs with a “dummy” pointing direction, each corresponding to a unique identifing number and lasting ∼ 15 days. In spinning mode, the mean pointing direction coordinates do not have a meaning. Rather ADC provides the mean Sun positions that determine the sky regions that are not accessible during each OB (∼ 60 degrees around the Sun and anti-Sun positions). Note

2 https://agile.ssdc.asi.it/current pointing.html The AGILE Data Center and its Legacy 7 that the allowed istantaneous pointing directions lie on a great circle orthog- onal to the Sun direction, whose orientation changes with time, so that the whole sky is accessible during a six months period.

5 Data Processing and Data Levels

The AGILE binary LV0 TM received at ADC is automatically archived after a consistency check, and transformed in level-1 FITS format (LV1) through the AGILE Pre-Processing System (TMPPS) [3]. The Level-1 data are then processed using the scientiﬁc data reduction software tasks developed by the AGILE instrument teams and integrated into an automatic quick-look pipeline system developed at ADC.

5.1 ADC Quick look: real time data processing

As soon as the Level-1 FITS files of the AGILE contact are produced by the TMPPS and archived, the data processing involves the following steps: 1) A first step (CORRECTION) on a contact-by-contact basis, aligns times all Telemetry to Terrestrial Time (TT) and performs some preliminary calculations and unit conversions. The resulting archive of FITS files is called LV1corr. 2) In a second step (QLSTD) for GRID Telemetry Data an ad-hoc implementation of the Kalman Filter technique is used for track identification and event direction reconstruction in detector coordinates. Subsequently, a quality flag is assigned to each GRID event: (G), (P), (S), and (L), depending on whether it is recognized as a γ-ray event, a charged particle event, a single-track event, or if its nature is uncertain (limbo), respectively. An AGILE auxiliary data file (LOG file) is created, containing all the information relevant to the computation of the exposure, live-time, orbital and attitude reconstruction. Then the AGILE event files are created, excluding background events flagged as particles, also reconstructing the event direction in sky coordinates. A QLSTD process, recently optimised in order to reduce latency, starts as soon as all the necessary input telemetries (GRID, Star Sensors, GPS, Housekeepings) have a common minimum time coverage. 3) As a third step, the ADC Scientific Quick Look Analysis (QLSCI) is run. Counts, Exposure, and Galactic background γ-ray maps are created pop- ulating the QL Level-3 (LV3) archive. To reduce the particle background contamination only events flagged as confirmed γ-ray events are selected, excluding observations during the South Atlantic Anomaly (SAA). To reduce the γ-ray Earth Albedo contamination, events whose reconstructed directions form angles with the satellite-Earth vector smaller than 80 degrees are rejected. The ADC QLSCI processing results in a set of Daily 8 Carlotta Pittori, on behalf of the AGILE-SSDC Team

Fig. 6 A screenshot from the “AGILE Services”: a web interface to the Mission Flexible Database relational DB.

Reports with selected candidate detections which are sent via email to the AGILE Team twice a day. The ADC results of the steps 1) and 2) are also promptly forwarded to the AGILE Team site at INAF-OAS in Bologna (previously known as IASF-BO), where the AGILE Team Science Alert System (SAS) pipeline runs, generating alerts on a contact-by-contact basis with independent ﬂare search algorithms. These alerts are sent via SMS, e-mail, and through the notiﬁcation system of the dedicated App for smartphones and tablets AGILEScience [2].

5.2 ADC Standard Analysis and Consolidated Archive

At the end of each AGILE Observation Block, the real-time quick-look archive used for alert generation, is consolidated by some completeness checks and possible reprocessing. Then the ADC Standard Analysis is run on each OB, producing the official AGILE-GRID level-2 EVT and LOG archive (LV2STD), compliant with the Office for Guest Investigator Programs (OGIP) standards recommended for FITS files, to be used for all the scientific publications, except for GCNs and Atels alerts. Data to be distributed to the Guest Observers are extracted from LV2STD event files, including γ-ray events in a region of 15 degrees radius around the requested source positions.

6 The “AGILE Services” and the AGILE Storage at ADC

Management and control of processing pipelines data and the corresponding outputs is entrusted to a management system developed by the IT support of SSDC, including the Common Pipeline Control Subsystem. A Flexible The AGILE Data Center and its Legacy 9

Database Web Interface (password protected) to the AGILE Mission relational database (MySQL DBMS), called “AGILE Services”, allows to rep- resent hierarchical relationships between the data, to get information about the automatic pipeline processes, to navigate between the connected data, to read, insert, modify, and retrive archived data. Different kind of accredited users (administrators, internal ADC operators, AGILE Team scientist, Guest Observer) have access to different functionalities: Data Mangement, Pipeline Management, Mission Planning, etc., see Fig. 6. A Proposal Management and Data Distribution system has been specifically developed for the AGILE Guest Observers, including an AGILE Target Visibility Computation Tool. The AGILE data storage at ADC increase at a rate of about 1 TB/year. The AGILE consolidated archive, including reprocessing and QL data contains about 10 TB at the time of writing.

7 AGILE Data Policy and Data Distribution

For the first seven years all AGILE data (i.e. both from the AGILE Team Projects and from the Guest Observer Program) have been subject to the proprietary rules normally applied to observatory space data: one year proprietary period after which they have been available via the public AGILE Data Archive at the SSDC. The one-year proprietary period started from the date when the Guest Observer or the AGILE Team received the data in a format that is suitable for analysis and publication. During the extended lifetime of the AGILE mission, a change in AGILE gamma-ray scientific data policy, proposed by the Mission Board, has been approved by ASI to strengthen the engagement of the Scientific Community by eliminating the one year proprietary period requirement. Starting from Oc- tober 2015 all AGILE-GRID data are published as soon as they are processed and validated. The new public AGILE archive now contains all data from De- cember 1, 2007 up to May 31, 2019, i.e. from OB 4900, start of Cycle-1 up to OB 32000 of the on-going Cycle-12. Public AGILE data are available from the ASI Space Science Data Center Multimission Interactive Archive (MMIA) webpages.

7.1 Public Scientiﬁc Software

For refined GRID scientific analysis a new AGILE public scientific software package (AGILE SW 6.0) adapted from the AGILE Science Tools developed by the AGILE Team (TAGNAME = BUILD25) is available from the ADC webpages. It includes new scientific software tasks and calibrations, an updated model for the Galactic diffuse gamma-ray emission, a refined procedure for point-like source detection, and the search for extended gamma-ray sources. 10 Carlotta Pittori, on behalf of the AGILE-SSDC Team

Fig. 7 Online version at SSDC of the Second AGILE Catalog of Gamma-Ray Sources [7]. The interactive web table allows to access additional data, including AGILE skymaps, the broad band spectral energy distributions through the SSDC SED-Builder tool and the ﬁnding charts through the SSDC Data Explorer tool.

8 AGILE Catalogs and ADC Interactive Webpages

Nine AGILE catalogs have been published for the AGILE Mission at the time of writing this paper: four AGILE-GRID γ-ray source catalogs [4,5,6,7], three AGILE-MCAL Terrestrial Gamma-Ray Flashes (TGF) catalogs [8,9,10], one AGILE-MCAL GRB catalog [11], and one SuperAGILE source catalog [12]. For each published AGILE catalog an online ADC version is available in the form of interactive SSDC webtables, e.g. Fig. 7. The interactive ADC catalog tables, often providing supplementary material to the published papers, have many functionalities and also provide access to the multimission SSDC scien- tiﬁc tools (cross-search, sky-region data explorer, spectral energy distribution tool, etc.). All the catalogs webpages are also being standardised for Virtual Observatory functionalities.

9 AGILE Legacy Archive and the AGILE-LV3 Web Tool

The γ-ray scientific analysis over long time scales may require long processing times. For example, to produce deep exposure maps over a time interval of about seven months, centered on the Crab Nebula position, can take at least 2 hours on 3 GHz Xeon CPUs. To speed up the AGILE-GRID scientific analysis, a complete level-3 (LV3) archive of pre-compiled exposure, counts and diffuse background maps over 1-day integration time, with standard parameters was created at ADC. The centers of the maps, their size and other relevant parameters are fixed, and have been chosen in such a way as to cover the whole sky. This AGILE-LV3 legacy archive can be used as basis for scientific Maximum The AGILE Data Center and its Legacy 11

Fig. 8 The SSDC MMIA query page for the AGILE-LV3 scientiﬁc analysis web tool.

Likelihood analysis on time scales that may vary from weeks to months, or even over the entire duration of the mission. For an easy on-line AGILE oﬃcial data analysis, the interested user may query the entire public LV3 archive through the AGILE-LV3 data analysis web tool, Fig. 8. In the query page the user can enter the source name or sky coordinates of the object he/she wants to analyze, the period of interest and the duration of the LV3 maps (e.g. 1, 7 or 28 days timebins) to be used in the analysis. The output from the query automatically selects all AGILE available observations of the source. The user can execute the standard AGILE Maximum Likelihood (ML) analysis by clicking on the Interactive Analysis buttons, or directly generate the γ-ray light curve over the selected period, with waiting times ranging from a few seconds to a few minutes (depending on the number of selected timebins). The AGILE-LV3 tool is meant to be easily comprehensible, does not require any install-on-premises software or calibrations, and it has been also tested with high-school students.

Acknowledgements This paper in written on behalf of the scientific team of the AGILE Data Center at SSDC, which is currently composed by: C. Pittori (coordinator), F. Lucarelli, and F. Verrecchia (deputy coordinator), with the main IT support of G. Fanari (Telespazio). Many people have contributed to the ADC in the past 10 years, including scientific staff and IT support staff, with variable FTEs depending on the operational phase of the Mission. We acknowledge the contribution of (in alphabetical order): A. Antonelli, S. Cutini, S. Colafrancesco, D. Gasparrini, P. Giommi, M. E. Pennisi, B. Preger, P. Santolamazza (scientific support); F. Acerra, P. D’Angeli, A. Guerra, D. Navarra, W. Oliva, R. Primavera, S. Stellato, F. Tamburelli (IT support). The ADC operate in close relationship with the Telespazio Mission Operation Center at Fucino, and in particular with the spacecraft operations manager (SOM) P. Tempesta. We also thank the AGILE Mission Board, composed of: the PI of the AGILE Mission M. Tavani, the Co-I G. Barbiellini, the current ASI Mission Director F. D’Amico, as well as the former Mission Directors: Luca Salotti, up to September 20, 2010, and Giovanni Valentini up to January 22, 2015, and the ASI representative E. Tommasi di Vignano. We would like to acknowledge the financial support of ASI under contract to INAF: ASI 2014-049-R.0 dedicated to SSDC. The paper is dedicated to the memory of the late Francesca Tamburelli. 12 Carlotta Pittori, on behalf of the AGILE-SSDC Team

Compliance with Ethical Standards

The author declares that there is no conﬂict of interest.

References

1. Tavani, M., et al., “The AGILE Mission”, A.&A., 502, 995 (2009) 2. Bulgarelli, A., et al., “The AGILE Alert System for Gamma-Ray Transients’, ApJ, 781, 19 (2014) 3. Trifoglio, M. et al., ‘Architecture and performances of the AGILE Telemetry Preprocess- ing System (TMPPS)”, Proc. SPIE, 7011, 70113E (2008) 4. Pittori, C., et al., “First AGILE catalog of high-confidence gamma-ray sources”, A.&A., 506, 1563-1574 (2009) 5. Verrecchia, F., et al., “An updated list of AGILE bright gamma-ray sources and their variability in pointing mode”, A.&A., 558, 137 (2013) 6. Rappoldi, A., et al., “Search of MeVGeV counterparts of TeV sources with AGILE in pointing mode”, A.&A., 587, A93 (2016) 7. Bulgarelli, A., et al., “Second AGILE catalogue of gamma-ray sources”, A.&A., 627, A13 (2019) 8. Marisaldi, M., et al., “Properties of terrestrial gamma ray flashes detected by AGILE MCAL below 30 MeV”, Geophys. Res. Space Physics, 119, 1337-1355 (2014). 9. Marisaldi, M., et al., “Enhanced detection of terrestrial gammaray flashes by AGILE”, Geophys. Res. Lett., 42, 94819487 (2015) 10. Marisaldi, M., et al., “On the High-Energy Spectral Component and Fine TimeStructure of Terrestrial Gamma Ray Flashes”, Journal of Geophys. Res.: Atmosph., 124, 7484 (2019) 11. Galli, M., et al., al.,“AGILE mini-calorimeter gamma-ray burst catalog”, A.&A., 553, A33 (2013) 12. Feroci, M., et al., “Monitoring the hard X-ray sky with SuperAGILE”, al., A.&A., 510, A9 (2010)