Name and Adress

Background document EIONET NRC Information Systems Workshop 27-28.11.08

Type: Working paper Date: 21 十一月 20088 Authors: Bernt Röndell, Jan Bliki, Søren Roug, Chris Steenmans - IDS

Subject: Evolution of EEA/Eionet data & information flows

I. Introduction

The main scopes of SEIS is about getting access to and sharing data and information that is handled in information systems and in organisations all over Europe, both at local, regional, national as well as at the European level (see also http://ec.europa.eu/environment/seis/index.htm ). Three of the seven guiding principles as listed in in the Communication (COM(2008) 46 final) are especially relevant for the discussion on standards and interoperability, namely:

- information should be managed as close as possible to the source - information should be readily accessible to end-users (primarily public authorities at all levels local to European) to enable them to access timely the state of the environment, the effectiveness of their policies and design new policy - information sharing and processing should be supported through common, free open-source software tools.

One of the operational objectives for the EEA SEIS work programme in 2009- 2013 is to improve sharing and access to the data that are stored in the organisations involved in Eionet. The ambition is to gradually extend this concept to the many existing information providers at local, regional, national, European and global level. We know that there is not one single model for the organisation of all data flows – the most effective and timely construction of the information flows needs to be developed according to the needs in each of the topic areas. That development should have as a base line the properties of each dataflow. EEA is planning to gradually move from the current system based on reporting data flows towards online live information systems.

Reportnet aims currently to handle about 500 data flows. That figure is in itself a proof of success but behind that figure a number of different kinds of data flows exist (see deliveries catalogue A, B below). Most of them (probably around 80 % of them that are periodical in nature) will in the future be reported through the modified Reportnet, some of them will still be reported through questionnaires and still some (where on line systems exist both at the national and European level) will be delivered / accessed through automatic processes that results in near real time update between national and international databases (like Ozone web).

Many of the national organisations have integrated quality control and validation functions into their on line systems (Described under C below). At the NRC IS-

D:\Docs\2017-12-14\0cf6230c2f06113bbd3df119cc8f8cd3.doc meeting in October 2007, the NRCs stressed their interest in having connections towards those systems directly instead of exporting data packages and manually cross check such deliveries. Deliveries described under C below are the only type of deliveries that we can move into a higher level of automation and also towards near real time without heavy and costly investments.

The different models (A to E) for future data exchange within SEIS will be presented at the Eionet Workshop to provide a framework for discussing the need for open standards to move towards improved interoperability between environmental information systems.

II. Catalogue of Deliveries

A. Deliveries based on Questionnaires

National Reporting / Reportnet International 1) Reportnet questionnair es. Countries fill forms directly into reportnet

Operator Quality Filling Questionairs Forms Directory ROD CDR Data Conversion DMM Packaging Warehouse Dictionary Services Assurance

Description The National level use forms inside Reportnet to deliver data. In practice this means that the moment it is time for delivering the data / information a person will log in on Reportnet and manually fill in a questionnaire. The result on international level is small datasets that are serving the purpose of compliance reporting. But in some cases (OECD Questionnaire) it also used for creating information. That procedure however has been questioned by the MS asking for another kind of procedure. Facts - A small number of records from every National body. - Very aggregated data - Slow data flows that takes place at maximum once a year. - The data has no further value than the intended assessments originally planned. Example Reporting obligation for Natura 2000 Standard Data Form (Habitat) Impacts - Countries only invest a small amount of resources to collect and report this data. - Automated indicator assessment will not be possible. The detail of data is very low and does not allow any further analysis. - SEIS will have very little interest in those flows unless more details can get provided. However the Commission have a high interest. Actions Reportnet. User dialogue with the COM?

D:\Docs\2017-12-14\0cf6230c2f06113bbd3df119cc8f8cd3.doc Page 2 B. Deliveries based on offline applications National Reporting / Reportnet International 2) Reportnet data deliveries . Countries maintain an offline application and export for delivery in reportnet

Quality Data Quality Offline systems Packaging Directory ROD CDR Conversion DMM PackagingWarehouse Assurance Dictionary Services Assurance Description The National level keeps a database designed for one or more international obligations. They maintain those databases offline on regular intervals. Those databases are then packaged and passed on to Reportnet. The main reason for countries not making this as online systems is in most cases because of the time intervals that are relatively low and because the gathering of the data from lower levels is done manually. The cost/benefit of this systems is lower when done offline than online. Facts - Long manual process to produce the dataset. - Small additional records towards a previous collected list. (Time series) - Relative low time interval makes it possible to put one person on the job for a short time. (Example here is one month of work to produce the dataset form other sources.) Example Corine land cover. A process that is based on manual interpreting satellite data. This process is on every 5 year intervals and is project based. Project based means that a new team is put together to generate a new update. Bathing water. Most countries are gathering the data during summer period and only for the days people go bathing to the beach, lakes or rivers. Most information is initially gathered manually and at the end of the season entered as yearly averages into a database. Impacts - Countries investment is higher than in case A. But because of it nature can not be done according the solution of case A. - Automated indicator assessment could be possible if the indicator can live with slow data updates for this data flow. SEIS will have little impact on those flows. The potential to move into online systems is greater but this need to be a checked on a case by case situation. Actions Reportnet. Moving offline applications into online systems on national level is a slow process which will take several years and must be looked at on case by case basis to ensure that it is possible and cost effective. However, EEA should investigate how to handle these flows when that happens. (Some MS will invest in online services for their public and EEA should find a way to harvest them). Create a process that makes data into information (IMS) at the European level (an information service)

D:\Docs\2017-12-14\0cf6230c2f06113bbd3df119cc8f8cd3.doc Page 3 C. Deliveries based on online systems

3a) Reportnet data deliveries . Countries maintain an online systen and export for delivery in reportnet

Online systems Quality Assurance Directory ROD CDR Data Conversion DMM Quality Packaging Warehouse Packaging Dictionary Services Assurance

System maintanance Description The National level maintain an online system that either collects data directly from stations or automatically collects data from regional systems or is a central system where everybody logs in and make there changes online. Online systems expose this information directly to the internet (open or secured). Those systems have a team of people who maintain the system. Typical behavior of online systems is that the data can be changed at any time. Facts - Large amount of data. (ex. Monitoring Stations) - Changes can be provided at any time. (Ex. Industry reporting to PRTR) - Any system that is based on monitoring stations is most probably based on online systems. The only issue might be that they are not always connected to the internet. - Any centralized National system could become or is already an online system. Example Ozone: All countries have an online system and EEA is using those inside Ozone web. The is a first of it’s kind and to be SEIS complained it should allow other international or national organisations to make use of that same setup. The concept proves that near real time is technological possible. PRTR: Most probably all countries did move or are moving this into an online system because the cost to maintain a website and gathering the information will be lower than doing this offline. Impacts - High impact on resources. Most of the time we talk of a team of people. The system is the cheapest solution for the necessary and required data flow. - This is a technological driven move from Case B to C. The cost of implementing this in a concept like B would be much higher for the simple reasons of the update intervals that are very high. - SEIS will have a big impact on those flows but the datasets could be very valuable in an assessment process when they are seen as constants i.e that are just slowly changing over time. Actions Move towards an on line delivery accessible for the European level (D1). Develop Discovery services that is based on a metadata standard that “goes beyond Dublin Core (INSPIRE metadata specifications). Create a process that makes data into information (IMS) at the European level (an information service)

D:\Docs\2017-12-14\0cf6230c2f06113bbd3df119cc8f8cd3.doc Page 4 D. Deliveries through online system

3b) Delivery happens near real time. ~6 months - 2 year Both systems are monitored . (SEIS) ~Near real time Data flow consortium Quality Packaging Warehouse Assurance ~Near real time Data Protocol Online systems Data Online systems Conceptial data flow model Protocol

System System Data Near real Services maintanance maintanance Protocol time DB Description In this setup on line deliveries are replaced into life systems on both the national and the European level. Both sides have a team that maintain and manage those flows. Both flows can be automated only when there is well established connection between those teams. Because of the statements above; near real time is the only way data can be transported. Systems with a lower flow rate would become more costly and might not have the right amount of resources available to maintain a fully developed system. In practice this means that those data flows that fit into this group will become packaged at international level. But also that the countries have their own packaging process in place. Facts - Near real time update between National and Inter-national databases. - Reduce manual costs related to package reporting. This is only true when such national life systems exist. - Possibilities for near real time European services and reporting. - Possible data flows between anybody including regional to regional from different countries. - Possible return of European services to any national or regional system - Common flows towards other international organisations. - More data than originally requested inside an obligation. (Could be the sum of several obligations from different international organisations) Countries have shown interest in participating – opening up their systems – for this kind of delivery. Example Ozone web could be seen as one of the first setups for EEA. Ozone web is now looking in how it can be open to other international organisations and pass on the near real time data further. Ideal would be if that is done already from the source. Strong evolvement from the providers will be needed to get this happening and that could be accommodated by a data flow consortium Impacts - Higher resource impact at International level - Need for technical communications between the teams hosting online systems dealing with the same data flow. (Data flow consortium (See below)) Actions Start the project that moves processes from C towards D. (See below) Create a process that makes data into information (IMS) at the European level (an information service)

The “Data flow Consortium” is needed when the data flows will affect more international institutions than EEA. The task is to discuss technical common solutions in order to facilitate and maintain the data flow specified. E. Deliveries using Sensor Web

D:\Docs\2017-12-14\0cf6230c2f06113bbd3df119cc8f8cd3.doc Page 5 ~6 months - 2 year 4) Sensor web Sensors alert when treshold occurs ~Near real time

Quality Packaging Warehouse Assurance Online ~Near real time Data Protocol systems Online systems Data Protocol

System System Data Near real Services maintanance maintanance Protocol time DB Description The future data flows based on monitoring station could evolve in sensor web systems. Intelligent sensors only send messages when thresholds are exceeded. The European and the national level can use the same sensor network and maintain different threshold settings to accommodate there needs. Facts - Very large networks which only receives information above the preset threshold. - Can only be based on online systems. One alert can happen at any time and any place. - A strong need for communication and standardisation between stations and online systems. Example - No examples known at the European level so far. Impacts - Higher resource impacts on International level all other options. - Strong need for communication and agreements to those who manage that sensor network. - Excellent setup for alert systems. Actions Follow outcome of FP6/7 RTD projects related the use of sensor web for environmental monitoring.

D:\Docs\2017-12-14\0cf6230c2f06113bbd3df119cc8f8cd3.doc Page 6