ERAD 2012 - THE SEVENTH EUROPEAN CONFERENCE ON RADAR IN METEOROLOGY AND HYDROLOGY

EUMETNET OPERA Radar Data Centre - Odyssey - An operational service and future plans.

Stuart Matthews 1, Nicolas Gaussiat 1, Robert Scovell 1, Daniel Idziorek 2, Karine Bouyer 2 1Met Office, FitzRoy Road, Exeter, EX1 3PB, , [email protected] 2Météo , 42, avenue Coriolis, 31057 Toulouse, France (Dated: 29 May 2012)

Stuart Matthews

1. Introduction The main objective of the third EIG EUMETNET OPERA (Operational Programme for the Exchange of weather RAdar information, 2008-2012) is the development and operational running of a European Radar Data Centre, Huuskonen et al. (2012). The development of Odyssey builds on previous OPERA work to develop a Pilot radar data hub. The Pilot radar data hub was developed and hosted by the UK Met Office and ran operationally from 2006 through to January 2012, when it was replaced by Odyssey, Harrison et al. (2006). Having verified that the concept of centralised processing was viable, EUMETNET gave the go-ahead for the OPERA Data Centre (ODC) project. The project started in 2007-2008 by gathering user requirements and producing a technical specification for the outputs, Chèze et al. (2009). In spring 2009, EUMETNET approved a joint Météo France – UK Met Office bid to build Odyssey and development started in autumn 2009, Dupuy et al. (2010). The first operational products were disseminated in January 2011 and the system was given full operational sign-off and acceptance by EUMETNET in April 2011.

Odyssey currently produces, every 15 minutes, European composites of surface rain-rate, maximum reflectivity and hourly accumulation at 2km resolution achieving above expectations timeliness and availability, thanks to the built-in high level of operational resilience provided by an original design: a virtual hub shared between Météo-France and the UK Met Office.

This paper presents the results of the first full year of the operational service along with the plans for Odyssey development during the final year of OPERA III and the next 5 year OPERA project (2013-17).

2. The Operational Service The challenges overcome in the initial development phase were: 1) to maximise the performances of the software, both in terms of CPU and memory used so that raw data from up to 200 European radars can be decoded, processed, projected and composited in less than 5 minutes. 2) to guarantee a high-level of accuracy in the compositing method used so radar products of homogenous quality can be produced for the whole European domain. 3) to establish a single information model to be applied to all incoming data 4) and to get all the data to Odyssey from National Meteorological Services in a timely fashion.

2.1 Operational Configuration The operational Odyssey service runs in parallel at the UK Met Office and Météo France operational centres in Exeter and Toulouse respectively. Both centres receive and process the same raw data using the same algorithms to create identical composites. The only difference between the two centres is only one is disseminating products at any given time. This cross organisational support gives the Odyssey service a very high level of resilience, Fig 1.

Moving all the raw radar input data and composite products around Europe, and also between Toulouse and Exeter, requires a robust IT communications network. The WMO Global Telecommunications System (GTS) within Europe (RMDCN or Regional Meteorological Data communication Network) delivers such a network as it provides a very high level of guaranteed service. Therefore, all data traffic between Toulouse and Exeter travels via RMDCN. National Meteorological Services have also been encouraged to supply raw data products to Odyssey using RMDCN. Unfortunately there are bandwidth constraints within RMDCN and most NMS have therefore chosen to supply data via the Internet. As National Meteorological Services upgrade their bandwidth on RMDCN we hope to see an increased use of RMDCN and therefore an increased level of resilience in the data supplied to Odyssey.

Odyssey is now established as a Data Collection or Production Centre (DCPC) of the WMO Information System (WIS), and Odyssey input data (and QC flags) and composite products will eventually be made available to a large number of users from the NWP, Aviation and Hydrology communities. ERAD 2012 - THE SEVENTH EUROPEAN CONFERENCE ON RADAR IN METEOROLOGY AND HYDROLOGY

NMS NMS

Odyssey System

Primary

Primary Toulouse Exeter

Message Switch

Composite User

Fig. 1 Odyssey IT communications routes configured with Toulouse as the ‘operational’ node

2.2 Performance Since the operational service started in January 2011 the timeliness and availability of Odyssey composite products has been maintained at a very high level and has consistently exceeded users Service Level Agreements (SLA), Table 1. SLA: 1) Availability: The percentage of composite products made available to DWD – Target 99.0%. 2) Timeliness: Composites delivered within 20 minutes of data time (DT+20) on at least 95% of occasions.

Month Availability Timeliness Month Availability Timeliness % (

2.4 Products Currently Odyssey produces three composite products: 1) Surface rain rate: weighted average of the valid pixels of each contributing radar, with weight having exponential dependence on the beam altitude 2) Max reflectivity: maximum of all the available local pixels values 3) Hourly rainfall accumulation: sum of the four 15 minutes previous rainfall intensity composites.

Domain Whole of Europe (see “Fig. 2”) Projection Lambert Equal Area Update frequency Every 15 minutes Spatial resolution 2 km Issue time Approximately 15 minutes after data time Format HDF5 or BUFR Table 2 Odyssey composite characteristics

Fig. 2 Sample Odyssey Max reflectivity composite

2.5 Data processing To produce a homogeneous product for Europe a set of pre-processing algorithms and quality measures needs to be agreed and then applied to the incoming data. At this time, pre-processing is in its infancy on Odyssey and we expect to see significant improvements over the coming years. Currently, only very basic pre-processing is applied on Odyssey. These are currently limited to: • A consistency check of the received scan against the details stored in the Odyssey database. If a scan is not found in database, the details of the new scan are added to the database and some pre-calculations are performed to allow the composite algorithm to be more efficient when subsequent scans are received. • A simple estimate of the quality of the radar observation is produced, for each range gate, which is based on the ERAD 2012 - THE SEVENTH EUROPEAN CONFERENCE ON RADAR IN METEOROLOGY AND HYDROLOGY

height of the radar beam. The quality value is given by u(r) = exp ( - alpha * ( h(r) - h_0 ) ) where h(r) is the height as a function of range r from the radar, alpha is a constant chosen such that u(h=1000.0 m) = 0.5 u(h=h_0) and where h_0 = 200.0 m is a threshold height above the radar, below which the radar observations are rejected. • A file containing the detection count for the received scan is updated. These files are kept on a rolling monthly cycle. It is the intention to use the detection counts to identify and exclude permanent clutter from the received data. Further work is required to determine a suitable rejection threshold (percentage of the maximum number of counts) in order to apply the filter. Once this algorithm is implemented, it will be possible to exclude cluttered pixels from the composites.

3. Future Developments While the current operational composites only have basic data pre-processing applied, future Odyssey products will incorporate additional centrally generated quality. Using expertise within the OPERA and BALTRAD communities, the additional pre-processing is being further upgraded to create the higher value composites and products needed for severe weather development monitoring and initialising and verifying meteorological and hydrological forecasts. 3.1 User requirements During the Summer-Autumn 2011 OPERA gathered requirements from NWP centres within Europe, Bölöni (2011). Results were returned from 19 National Meteorological services (, Czech Republic, Germany, , France, , Ireland, , the Netherlands, Norway, Poland (ALADIN), Poland (COSMO), , Romania, Russia, Spain, Sweden, and the United Kingdom)

The returns concluded: • that the European NWP community is very much motivated to assimilate radar data into fine scale models. • Odyssey is the ideal framework to manage radar data exchange for the above purpose. • However, presently only 2D composites are available, which does not satisfy the needs of data assimilation • European models require 3D volume data to feed their assimilation systems. • Data quality control or indication is an important issue to data assimilation. The intention of getting 3D volume data with quality control/indication is thus articulated towards the OPERA management by the NWP community. 3.2 Plans 2012 Although the ODC development project fully met its mandatory requirements and all its agreed objectives, including delivery of an operational service on time and to budget, it did not include the necessary modules to create high quality radar products. At the start of the project it was assumed a sufficient quality level could be provided in all volume data sent to Odyssey, e.g. cleaned of ground clutter at the source. However, all OPERA members do not have such data, and it has also appeared that many data sets are contaminated by external transmitters, such as RLAN.

The insufficient quality of Odyssey products for modelling prompted EUMETNET to release additional funding to OPERA to improve the quality during the present programme phase. The April 2011 OPERA meeting decided that the first priority is the removal of the non-meteorological echoes from the incoming volume data. This would be done by implementing modules from the Baltrad toolbox into the Odyssey system. Following software integration testing and demonstration of an improvement in data quality, implementation of agreed modules (anomaly detection algorithms developed at the Finnish Meteorological Institute and operational implementation of a clutter filter) is about to take place ready for operational release before the end of 2012. 3.3 Plans 2013-17 All the EIG EUMETNET observations programmes, including OPERA, are currently going through a review process ready for the next 5 year phase (2013-17). This process has included gathering user requirements and assessing the available budgets. The mandatory requirements for the next phase of OPERA related to Odyssey include: • Maintaining the Odyssey operational services to the same performance as present system. • Support to data suppliers for encoding/decoding in HDF5 and BUFR formats, and for keeping updated the OPERA Data Information Model (ODIM), and the associated HDF5 structures and the local BUFR tables • Adding of centrally produced quality information to the incoming data at the pixel level. Flagging various classes such as hydrometeor echoes, ground clutter echoes, clutter corrected pixels, pixels at noise level, pixels suffering from strong attenuation, pixels suffering from blockage etc) or of numeric type (amount of attenuation, blockage or clutter correction etc) represented as a suitable quality index. The quality information is to be used in ODYSSEY composite production and included in volume data (reflectivity and radial wind) distributed to modellers. • Improvements to the compositing algorithm so that flagging of data pixels and the other quality information on the volume data is taken into account in the compositing, and that the composites are delivered with quality ERAD 2012 - THE SEVENTH EUROPEAN CONFERENCE ON RADAR IN METEOROLOGY AND HYDROLOGY

information. • Addition of a system to distribute flagged radar volume data, both radar reflectivity and the radial wind, to the modeling consortia and other radar data users. No filtering or smoothing to be applied to data to be distributed to the NWP users, except the addition of quality information.

At the time of writing, bids are being prepared for the next phase of OPERA and a final decision will be announced in Autumn 2012.

Acknowledgments All National Met Services who have provided their data to Odyssey are thanked for their efforts. Without their contribution, the success of this key OPERA programme objective would not be possible. Thanks must also go to all national delegates to the OPERA programme for their support and advise through the development project and operational service.

References Chèze, J-L., Hafner, S., Holleman, I., Matthews, S. & Michelson, D., 2009 Specification of the EUMETNET operational Weather Radar Data Center. OPERA deliverable OPERA_2008_02. Dupuy, P., Matthews, S., Gaussiat, N., Scovell, R. & Kergomard, A., 2010 Developing a European Radar Data Centre. Proceedings 6 th European Conference on Radar in Meteorology and Hydrology, Sibiu. Bölöni, G,. 2011 A summary of questionnaires on the NWP requirements towards OPERA related to the possible future re-transmission radar 3D volume data. OPERA working document WD_2011_09. Harrison, D. L., Scovell, R. W., Lewis, H. W. & Matthews, S. J., 2006 The development of the EUMETNET OPERA radar data hub. Proceedings of the 4th European Conference on Radar in Meteorology and Hydrology, Barcelona. Huuskonen, A., Delobbe, L. & Urban, B., 2012 EUMETNET OPERA: Achievements of OPERA-3 and challenges ahead. Proceedings 7th European Conference on Radar in Meteorology and Hydrology, Toulouse. Michelson, D.B., Lewandowski, R., Szewczykowski, M., Beekhuis, H., 2009 EUMETNET OPERA weather radar information model for implementation with the HDF5 file format. OPERA deliverable OPERA_2008_03.