Earth Observation Data and Knowledge Preservation at ESA The

Summary

Earth Observation data and VThe Earth Science world and Earth knowledge preservation at ESA Observation capabilities from bit archiving to Earth Science long term applications VEO operational scenarios VEO archiving, preservation and exploitation issues

Vinzo Beruti & Luigi Fusco ESA - Earth Observation Programmes

e-IRG - User needs in IT and Information Infrastructure Nottingham, 6 Dec 05

The Earth Science world Earth Observation potential

Coupled and inter- GOME total ozone assimilation disciplinary processes Stratospheric Ozone

e r e h p Industrial Emissions s o tm A Cryosphere Complex web of sensor Atmospheric profiles

Biosphere

n a e c O Topography & Motion

Land cover & vegetation Optimal Trajectory First guess Marine SST, SSH& colour

Field Currents, bathymty & ice

Noisy Complex data analysis observations 10 y displacement of Etna 1992-01 time

More Earth Observation potential European EO space capacity

1990 2000 2004 2010 ROMA IKONOS-1 Meteo Optical METEOSAT METEOSAT Second Generation M-1, 2, 3, 4, 5, 6, 7 MSG-1, -2, -3 image in cooperation with METOP-1, -2, -3 EUMETSAT

CryoSat (Polar Ice Monitoring) Science (Gravity and Ocean GOCE Circulation Explorer) For a better ADM/Aeolus comprehension of Earth the Earth System Explorers SMOS Earthcare SWARM ERS radar ERS-1, -2 ENVISAT derived Application Services imagery: Systems part of Global 10 y changes Earth Monitoring for Environment Lon term in ground and Security (GMES), environment Watch monitoring stability in cooperation with EC Earthnet : European access to non-ESA missions: European Landsat, SeaWifs, NOAA, JERS, MODIS, ALOS, Proba, Bird, Scisat... users

1 Some Earth Science digital data The Earth Observation operation scenario requirements V Wide distributed V Global, regional, local applications system – Alternative use of the data at different resolution environment: V Large historical distributed archives –multi-satellites – Long term data and knowledge preservation issues –multi-facilities V Near real-time access to data – For processing, value adding and dissemination V Integration with models to provide long term trends and forecast – Data assimilation V Integrate different data sources V Large user community – Standardisation, Virtual Organisation, … –multi-discipline V Need to link data to technical information and –research, institutional, scientific results commercial and operational – Need to keep/transfer the historical Knowledge –Large international partnership

A GMES example: The near future perspective: GMES Marine & Coastal Management

ng EMSA EU DGs EEA Int‘l conventions secretariats ivi rch es A tr OSPAR, Helsinki, MAP en C Coastal Space Coast Port Environme environme Health & Food/ guards authorities nt nt agencies safety fisheries observing Needs managers agencies agencies Airborne systems surveillance operators In-situ User observing oriented Systematic Decision Reference DB Coastal systems services monitoring and support administration Management detection for ALERT information information In-situ data collection Archiving AIS networks Centres Data integration Solutions VDS NRT NRT Ocean & information VTMS radar optical Meteo forecast state management Earth Scientists operators imagery imagery forecast models Governments Documents/ GS … Archiving EU International Organisations knowledge Documents/ Centres operator knowledge documents Regulatory Bodies Satellite s Meteo Ocean Industry operator models model http://www.gmes.info s Documents/ operators operators General Public knowledge

GEOSS – the wider view The EO infrastructure issues Global Earth Observation System of Systems

VBring together in the virtual organisation – the users: scientists, institutions … – the space data providers: acquisition stations, processing and archiving facilities – the other data providers VAccess services … not simple data – Interoperability issues VNeed for linking all available expertise and knowledge in data handling/usage

The interoperability framework

2 EO data volumes are increasing… Large no of facilities involved…

Development of ESA's EO Historical Data Archive between 1986-2004 •Historical missions • Non ESA missions • SPOT 1/3 AQUA Modis (April 03-today) facilities 1650 • Landsat 1/2/3/4/7 ENVISAT LR (March 02-today) •Active missions • Matera 1500 • NOAA serie • ESA missions ENVISAT HR (March 02-today) • ERS-2 (HBR, LBR) • Esrange This does not consider multiple 1350 TERRA Modis (June 01-today) • CZCS• ENVISAT (HBR,LBR) • I-PAFfacilities copies, and high level products! • SEASAT • UK-PAC 1200 QUICK SCATT (01-today) /PROBA (May 02-today) • SPOT2/4• Future missions • D-PAF LANDSAT 7 ETM (April 99-Dec 03) • JERS• LANDSAT1 SAR/OPS 5 (TM) • Maspalomas• UK-PAF 1050 • GOCE SEA STAR SeaWifs (Apr 98-today) • MOS1,1b • I-PAC • NOAA• 17/18 ADM (AVHRR) • Neustrelitz 900 ERS 2 HR (May 95-today) • NIMBUS • I-PAF • SCISAT• SMOS • Esrin ERS 2 LBR (May 95-today) • D-PAC 750 • PROBA • CRYOSAT • Tromso JERSAt SAR/OPS present VNIR (92-Sep (2005) 98) • TERRA • D-PAF 600 • Ifremer Total Archive in • Earthcare TerraBytes (TB) ERSthe 1 HR (Jultotal 91-Mar 00)volume of • E-PAC • AQUA • SWARM 450 ERSarchived 1 LBR (Jul 91-Mar data 00) and • CNES • QUICKSCATT•… SPOTproducts 1-4 HRV (87-today) (from ESA • IFREMER 300 • SENTINELS MOSacquired 1, 1b MESSR (87-Oct EO 93) missions) • LRAC 150 •… NOAAis 9-17>> AVHRR 500 (86-today) TB/y ) • Esrin 0 LANDSAT 5 TM (April 84-today) 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 NIMBUS 7 (Nov 78-May 86), SEASAT (Jun-Oct 78) VArchive facility coordination issues: Year LANDSAT 2-4 MSS (75-Dec 93) – Data format, media, archive systems, migration plans …

Some data archiving issues… Some data preservation issues….

V Volume of data to be archived are exponentially V Data integrity is not coherent across missions / increasing facilities V At present, ESA mandate is to maintain archives – Archive evolution - data migration on routine basis, for 10 years only after mission end of life format definition, media maintenance, technology evolution, security, etc. – For a missions like ERS-1, this period is already over. – Completeness - Integration of the physical telemetry V ESA is proposing a coherent technical and payload data as acquired by at the stations with the managerial strategy related ephemeris (orbit…), auxiliary data (processing – through Ground-segment coordination body with CNES, info to generate the nominal products…) ASI, DLR, EUMETSAT… – Logistic - where the data are physically stored – coordination efforts started with some practical results (processing may be performed in a separate location if V The long-term funding strategy is to be found required) – Space agencies use the funds from other on-going – access – information on what is archived and how to programme to maintain these historical datasets retrieve data (metadata, browse, products generation – Such a scheme is no sustainable on the long-term capability, products dissemination, local infrastructure, data reprocessing, etc.)

Actions taken for an European coordinated Some archive exploitation issues …. approach V Provision of reference consolidated data sets … – Systematic and periodical global dataset generation • The ESA strategy is based on principles defined in – Assimilation / integration processes (fill the gaps) • In agreement with ESA Delegation boards (DOSTAG, PB- – Science user support and acceptance EO – Technical issues: e.g. standards • PB-EO Ground Segment Task Force recommendations – On-demand access to simple and complex products • Coordination started at ESA + National facilities V Evolution of archive access via the G/S Coordination Group – Evolution of data processing algorithms • Common archive standards (SAFE) –… • Define common data model (OAIS) and operational procedures • Standardization of interfaces •…

3 Benefits for an European coordinated approach

V Cost reduction in terms of implementation and operations of existing and future projects – easier data exchange (based on same standards) – Better services to users Thank you! V Stronger European position at International level – Medium and Long term harmonized European data preservation policy – Leading voice in International Organizations, support coordinated approach also for GMES and GEOSS V Optimal definition of infrastructure for common [email protected] approach to data and knowledge preservation in [email protected] Earth Science – Easier share of Facilities and related infrastructure – Ensure preservation of historical data, after the end of the mission lifetime