EO Ground Segment & Data Access

ESA Missions and Sentinels ground segment approach

ESA ESRIN, GSCB Workshop 2007 19th June 2007 Content

1. ESA Missions 2. ESA Data policy 3. ESA Data products 4. ESA Ground Segment concept 5. User interface 6. Archive and Long-term data preservation strategy 7. ESA Interoperability Activities 8. Conclusions Mandate: EO data to Users

Manage and share ESA European EO Ground Segment infrastructure Earth Observation data users current and historic

Facilitate data access: European - On-line access Ground Segment - Near Real Time delivery Coherently manage facilities - Long term preservation ESA and ESA Third Party - Data reprocessing Non-European Mission Programmes - New technologies support Ground Segment interface and - Coherent data policy EO Data user facilities -etc…

Non-ESA Earth Observation data Worldwide (ESA Third Party Missions) users 1. ESA Missions 2. ESA Data policy 3. ESA Data products 4. ESA Ground Segment concept 5. User interface 6. Archive and Long-term data preservation strategy 7. ESA Interoperability Activities 8. Conclusions EO missions handled by ESA 1990 2000 2004 2010

METEOSAT METEOSAT Second Generation Meteo M-1, 2, 3, 4, 5, 6, 7 MSG-1, -2, -3 in cooperation with EUMETSAT METOP-1, -2, -3

Cryosat 2 (Polar Ice Monitoring) Science (Gravity and Ocean GOCE Circulation Explorer) to better understand Earth SMOS (Soil moisture) the Earth Explorers ADM/Aeolus EarthCARE SWARM ERS-1, -2 ENVISAT Applications ESA Sentinels satellites Services GMES + to initiate long term National missions (e.g. SPOT, monitoring systems Pleiades, TerraSAR, Cosmo-Skymed, ...) and services

Third-Party Missions: European access to non-ESA missions European ALOS, SPOT, Landsat, Kompsat-2, Scisat ... users ESA EO missions launch schedule e D/EOP Overall Launch Schedule

2006 2007 2008 2008 2010 2011 2012 2013 2014 2015 2016

MSG-2 21 December 2005 1st METOP 19 October 2006

Earth Expl. Core 1 - GOCE 1st quarter 2008

Earth Expl. Opp. 2 - SMOS May 2008

Earth Expl. Core 2 - AEOLUS June 2009 MSG-3 March 2009

Cryosat II March 2009

Earth Expl. Opp. 3 - SWARM February 2010 2nd METOP December 2010

GMES Sentinel 1 June 2011

GMES Sentinel 2 November 2011

GMES Sentinel 3 February 2012 MSG-4 June 2012

Earth Expl. Core 3 - EARTHCARE December 2012

GMES Sentinel 1b 2013 TBC GMES Sentinel 2b 2013 TBC

GMES Sentinel 3b 2014 TBC

MTG 2015 TBC METOP 3 2015

More missions are in the planning, yet not included due to not yet defined launch dates: Sentinel-4, Sentinel-5 and the 7th Earth Explorer 1. ESA Missions 2. ESA Data policy 3. ESA Data products 4. ESA Ground Segment concept 5. User interface 6. Archive and Long-term data preservation strategy 7. ESA Interoperability Activities 8. Conclusions ESA EO data policy

9 ESA has a single EO data policy for all its own missions (Envisat, ERS, Earth Explorers) and its Third Party missions.

¾ during the GMES pre-operations period, the GMES data access will be governed by specific Terms & Conditions.

9 The data policy for the Sentinel satellites is to be defined. 1. ESA Missions 2. ESA Data policy 3. ESA Data products 4. ESA Ground Segment concept 5. User interface 6. Archive and Long-term data preservation strategy 7. ESA Interoperability Activities 8. Conclusions EO data products at ESA

9 ESA usually provides data products to users up to Level 2 (CEOS definition).

¾ In some cases (user community requirements), additional product levels are provided: e.g. demonstration Level 3 products, Orthorectified Level 1 products.

9 When relevant, same processing algorithm/format are used between missions (e.g. ERS and Envisat).

9 Product calibration and validation are important activities to ensure products acceptance by user communities:

¾ Missions cross-calibration and cross-validation is essential for users ¾ ESA support for calibration and validation for Third Party missions when requested Product generation modes

Systematic:Systematic: •• Products Products areare generatedgenerated automaticautomaticallyally forfor allall receivedreceived rawraw datadata OnOn Request:Request: •• Products Products areare notnot generatedgenerated unlessunless spspecificallyecifically requesrequestedted byby anan useruser

NearNear RealReal TimeTime (NRT):(NRT): •• Products Products generatedgenerated withinwithin 33 hourshours fromfrom sensingsensing Off-LineOff-Line (continuity(continuity ofof servicesservices fromfrom NRTNRT toto Off-Line):Off-Line): •• Products Products generatedgenerated aa fewfew dadaysys toto weeksweeks fromfrom sensing;sensing; •• Similar Similar toto NRTNRT productsproducts (same(same format,format, samesame processingprocessing algorithmsalgorithms wheneverwhenever possible);possible); •• Benefit Benefit fromfrom anan improvedimproved knowledgeknowledge ofof auxiliaryauxiliary datadata (e.g.(e.g. orbit,orbit, calibration);calibration); •• Reprocessing: Reprocessing: whenwhen aa newnew procprocessingessing algorithmalgorithm isis availableavailable

UnconsolidatedUnconsolidated product:product: •• Product Product generatedgenerated inin NRTNRT usingusing availableavailable auxiliaryauxiliary datadata (i.e.(i.e. maymay nonott bebe mostmost preciseprecise orbitorbit vectorsvectors oror calibrationcalibration information).information). ConsolidatedConsolidated product:product: •• Product Product generatedgenerated off-lineoff-line usiusingng thethe mostmost precisprecisee auxiliaryauxiliary informationinformation available.available. ItIt isis timetime ordered,ordered, withwith nono overlapoverlap nornor datadata gap.gap. 1. ESA Missions 2. ESA Data policy 3. ESA Data products 4. ESA Ground Segment concept 5. User interface 6. Archive and Long-term data preservation strategy 7. ESA Interoperability Activities 8. Conclusions ESA GS strategic guidelines

9 Guiding the development and evolution of the PDGS for ESA handled missions ¾ Open & Operational ¾ Ground Segment Task Force recommendations

9 Key principles ¾ Ease data access ¾ Mission-independent Ground Segment • Envisat, ERS and Third Party Missions PDGS harmonisation • New concept for Earth Explorers PDGS implementation

¾ Competitiveness ¾ Coordination ESA multi-mission GS concept 9 N sources (satellites and/or archived data) 9 M users 9 Geographical distribution of functions among existing centres in Europe, competitively selected as per GS Task Force recommendations

Satellites Receiving Facility Sites Users Stations and Coordination Center ESA multi-mission PDGS concept

9 Single, distributed and shared

9 Network of Centres providing EO PDGS services to the missions operated by ESA and Member States

Generic Services Mission-specific services 9 User Services 9 (Re-)Processing 9 Archive, data retrieval, products 9 Instrument Planning generation , packaging and distribution 9 Instrument performance and Quality monitoring 9 Routine Product verification / QC ESA modular GS architecture

Elements can be installed in any Centre and multiple times (to enable geographical distribution)

Calibration, Validation and Verification User Access, Coordination and Control Calibration Parameter Monitoring and Acquisition and User Access Validation Management Verification Product Mngmt. Long-term User Instrument Product Verif. Mission Parameter + Information Calibration Planning Database Perf.Analysis Services

Order Management

Inst. Param. Interactive IPF Repository Master Order User Support and Auxíliary Product and Validation Catalogue Handling Help Desk Generation Analysis IPF IPF QC

Receiving Station Processing, Archiving and Dissemination Product Handling Request Handling

Station Data Library Production Online Data Archive, Site Cache, Control Local Inventory Control Access

Processing Reception Processing Ingestion SystemProcessing Dissemination System Circulation SystemSystem Circulation IPFIPFIPFIPFQCQC optional optional optional IPF IPF QC QC Decryption QC Reception Decryption QC Handling Processing Dissemination

Monitoring&Alarm Logging

Operating Reporting System Domain Functional Monitoring and Control Element Grouping GS elements management

9 Each element can evolve independently & individually with ¾ new mission requirements ¾ new technology 9 Single software maintenance for distributed elements 9 Standardisation where possible and of advantage (adoption of standards like SAFE) 9 Cost efficient data integrity through multiple archives 9 Introduction of a new mission ¾ identification of new functions as needed (e.g. encryption) ¾ modification of existing generic elements ¾ new mission specific elements (if needed) ¾ reconfiguration and AIV with the new mission 1. ESA Missions 2. ESA Data policy 3. ESA Data products 4. ESA Ground Segment concept 5. User interface 6. Archive and Long-term data preservation strategy 7. ESA Interoperability Activities 8. Conclusions Access to data catalogue (1/2) EOLI-SA: ESA multi-mission catalogue

t a..iinnt li..eessa //eeooli tttpp::// att hht bllee a aiillaab AAvva

Yesterday Access to data catalogue (2/2) EOLI-web: gateway to other EO data providers

g l..oorrg orrttaal ooppo ww..ee //wwww tttpp::// att hht bllee a aiillaab AAvva

e.g. DLR Collection s Access to on-line data

Currently different systems and interfaces providing direct access to EO data for download:

foorr accee f ts terrffa duuccts ctsts l iinnte prrood odduuc hiiccaal edd p t pprro raapph hhiivve ceennt eooggr aarrcc f rreec GGe TSRSR e oof S//AAT chhiivve ERIIS g aarrc MMER llliinng RRool

POTT r orr SSPO e ffoor pp ffo cttss rffaacce ckk-u-u dduuc inteter ctsts p ppiic pprroo caall in dduuc ftftp iveedd phhiic pprroo rcchhiv grraap issaatt aar Geeoog Ennvvi G enntt E rreecce

Objective: to simplify user interfaces for on-line data access Access to data in general

1-1- Physical Physical mediamedia Product CD-ROM, DVD-ROM, Discs…

Svalbard (KSAT) USERS 2-2- Internet Internet (HiSEEN)(HiSEEN) Tromsoe (KSAT) (KSAT) Farnborough Esrange (SSC) p Ukerna Uninet tt (Infoterra) Kiruna Salmijarvi h ttp (UK) (NO) r h (SSC) o r ftp o Sunet ftp RedIris (SE) Neustrelitz (ES) GEANT DFN (DLR) Maspalomas (DE) Oberpfaffenhofen (INSA) Renateur (FR) (DLR) GAR R (IT) Toulouse (CNES) Frascati MulticastMulticast (ESA) Matera (Asi, Telespazio) (Europe.(Europe. Africa,Africa, soonsoon America)America) 3-3- DDS DDS

Satellite-basedSatellite-based DisseminationDissemination 1. ESA Missions 2. ESA Data policy 3. ESA Data products 4. ESA Ground Segment concept 5. User interface 6. Archive and Long-term data preservation strategy 7. ESA Interoperability Activities 8. Conclusions ESA Long-term data preservation strategy

GOAL:

Ensure the long term preservation and the access of all ESA Member States EO acquired and repatriated data: ¾ Worldwide coverage of ESA and member states missions, ¾ European coverage or areas of European interest of non member states missions

for an unlimited time, as representing a unique, valuable, independent and strategic source of data to be delivered to the posterity. ESA EO Processing and Archiving

ESA has a distributed archive (holding ESA and non–ESA Missions: missions data) with a single multi-mission catalogue ENVISAT: F-PAC, E- • data are located in different facilities (about 12) PAC, D-PAC, UK-PAC, • reusing national archives and thematic expertise, federated Fin-CoPAC, I-PAC, ESRIN by ESA Tromsø ERS: F-PAF, D-PAF, UK- • enhance data safety with duplicated archives PAF, I-PAF, ESRIN Kiruna PROBA: Kiruna, ESRIN Sodankila

Terra/Aqua: Matera, Kiruna, Maspalomas, Oberpfaffenhofen

LANDSAT: Matera, Farnborough Kiruna, Maspalomas, Neustrelitz Neustrelitz Oberpfaffenhofen SCISAT: Kiruna Brest AVHRR: Tromsø, ESRIN Matera, Maspalomas Toulouse SEASAT: D-PAF Matera ESA Reference archive

National Archiving Maspalomas Centres shared by ESA Fast growing ESA EO data archive

Evolution of ESA's EO Historical Data Archives between 1986-2006

LANDSAT 2-4 MSS (75-Dec 93)

3000 AQUA Modis (April 03-today) 2850 2700 ENVISAT LR (March 02-today) 2550 2400 ENVISAT HR (March 02-today) 2250 2100 TERRA Modis (June 01-today) 1950 QUICK SCATT (01-today) /PROBA (May 02- 1800 today) 1650 LANDSAT 7 ETM (April 99-Dec 03) 1500 1350 SEA STAR SeaWifs (Apr 98-today) 1200 1050 Total Archive in in Total Archive

TerraBytes (TB) ERS 2 HR (May 95-today) 900 750 ERS 2 LBR (May 95-today) 600 450 JERS SAR/OPS VNIR (92-Sep 98) 300 150 ERS 1 HR (Jul 91-Mar 00) 0 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 ERS 1 LBR (Jul 91-Mar 00) Year SPOT 1-4 HRV (87-today) ESA long term EO data preservation activities

9 Archives Operations ¾ Archive Maintenance activities like logistic, data integrity preservation, data purging and compacting, data quality checks, etc. ¾ Archive Data Access that includes: • data exploitation • knowledge of what is archived and how to retrieve data (metadata, browse, products generation capability, products dissemination, local infrastructure, etc.) • evolution of processors (H/W, S/W), data reprocessing with new algorithms, homogeneity with new missions, etc. ¾ Archive Operations covers all manual or automatic activities to perform maintenance, data storage, data transfer, data processing and dissemination, data reprocessing 9 Archive Evolution ¾ ensuring evolution of formats, migration to new technologies, etc. Evolution of the ESA strategy: proposed approach 9 Following the draft GSCB Common Long-term Data preservation strategy – under discussion 9 It will be presented to DOSTAG during 2007

9 Data Logistic ¾ ESA to offer technology, sharing and interfacing to the national / industry archives ¾ Offer member states to insert national missions data beyond their national funding schemes into the distributed ESA archives, subject to related programme approval ¾ Harmonise interfaces and used technology to cope with the ever increasing number of distributed archives; ¾ Develop for and share technology with member states, inc. prototype advanced technologies ¾ Complement the European strategy through agreements with non European archives 9 Data Access ¾ Standard access to heterogeneous data: easy and coherent ¾ Offer special, easy data access conditions (enhanced category 1 scheme) for historical data ¾ Coordinated processor and technology evolution ¾ Coordinated re-processing schemes 9 Data Integrity ¾ Maintain data at recent technology via routine conversion, with information extraction capability ¾ Standard archive format definition and adoption ESA archives evolution areas

9 Archive architecture structure ¾ Distributed multi-mission archives networking fully transparent to users ¾ data processing/reprocessing (e.g. Large CPU clusters for fast, GRID-based) ¾ Maximum reduction of user access time latency (e.g. large bandwidth, consumer communication technologies adoption) ¾ Internet based user access (web based access) relying in interoperability standards, implementing the required security 9 Storage full operation automation 9 Adopted data archive format: SAFE - Standard Archive Format for Europe ¾ Advanced data description languages (public standards) ¾ being adopted for all ESA handled missions ¾ Attracting attention at European level (e.g. DLR for TerraSAR-X as exchange format), by CNES for Pleiades and other organizations, like USGS, OGC, CEOS). ¾ Could be suitable as product formats harmonisation standard 9 Media Storage technology ¾ High capacity and fast access tapes ¾ Large magnetic Disk-based archives ¾ Hierarchical archives organization for faster user access based on data characterization ¾ Holographic technology, Blue ray based technology, data compression 1. ESA Missions 2. ESA Data policy 3. ESA Data products 4. ESA Ground Segment concept 5. User interface 6. Archive and Long-term data preservation strategy 7. ESA Interoperability Activities 8. Conclusions IP

E R U T C U R T S A N R F

S I E I C V R E S HMA INFEO – EOLI-XML USNG

MMFI SAFE User Services User k ion nation i cts harmonisation logue SSE ta Acquisit Ca Help Des Web Portal Dissem Data Ordering CEOS Cal/Val WS Cal/Val Portal GECA Test sites characterisation Produ GAMME ANGLE Network Network FACILITY FACILITY e tion v i c

ch n o Ar Produ ssi i M Cal/Val Planning

Easy data access: Interoperability

Quality Control

Control

ng & & ng i ORDINATION CENTRE Monitor ORDINATION CENTRE - - CO CO HMA SaVoir CoMu MPAS Interoperability context

Using HMA Service Taxonomy – Discovery interoperability points

Catalogue • Discovery (collection, services) • Catalogue (earth observation Gen Public Order products) • Order (archive, subscription, on- Programming Science line access) • Programming (coverage order for Data Access Commercial future acquisitions, mission Interoperability bus GS1 planning feasibility) GMES • Data Access (web coverage server, web mapping server) As above • User Identity Management

GS2, GS3 … • Service Orchestration • Monitoring & Control ESA Historic Interoperability

CIP/Z39.50 based protocol Discovery Coordinated through Catalogue CEOS/WGISS

Gen Order Public • good interoperability between ESA/DLR at level of catalogue Science Programming

Data Access Commercial • flagging (almost no) interest Interoperability bus GS1 from other international partners GMES As above • increasingly “expensive” to maintain cf. newer technologies GS2, GS3 … ESA Current Interoperability

Web-Services based protocols Discovery with Open GIS Consortium and ISO “inspired” schema. Catalogue Developed ESA user service Gen Public Order and first proven with CNES.

Science Programming • good interoperability of online Data Access services between ESA/SPOT- Commercial Interoperability bus GS1 TPM at level of catalogue and GMES order As above • illustrated with other catalogues GS2, GS3 … (e.g. Kompsat) ESA Future Interoperability

HMA aligned to mainstream information technology (SOAP/SAML) Discovery and OGC standards (GML/CAT/SPS)

Catalogue Matured 9 Discovery Gen Public Order 9 Catalogue 9 Order Science Programming

Data Access Needs more study Commercial 9 Programming Interoperability bus GS1 9 Data Access / Web Coverage Server GMES As above DAIL implementation ITT issued HMA interface adoption by ESA GS2, GS3 … planned

HMA will be gradually implemented for ESA GMES pre-operations activities and for Sentinel satellites. 1. ESA Missions 2. ESA Data policy 3. ESA Data products 4. ESA Ground Segment concept 5. User interface 6. Archive and Long-term data preservation strategy 7. ESA Interoperability Activities 8. Conclusions Conclusions

9 ESA plans for a stepwise introduction of HMA, even if initial GMES pre-operational Data Access will use intermediate solutions.

9 ESA EO ground segments (Earth Explorers, GMES) will apply future GSCB recommendations.

Additional information: http://earth.esa.int