Earth Science Research at the Barcelona Supercomputing Center Dr. José M. Baldasano ([email protected])
Barcelona Supercomputing Center-Centro Nacional de Supercomputación (BSC-CNS) Earth Sciences Department. Barcelona, Spain
Mars, 2009 – Barcelona, Spain
A unique place ………… MareNostrum
10240 IBM Power PC 970MP processors at 2.3 GHz (2560 JS21 blades). 20 TB Main Memory. 94,21 Tflops (peak performance). 280 + 90 TB disk. Interconnection networks: Z Myrinet Z Gigabit Linux cluster (SuSe). Diskless network support.
MareNostrum
MareNostrum’s evolution:
World Europe List Position Position November 2004 4 1 June 2005 5 1 November 2005 8 1 June 2006 11 3 November 2006 5 1 June 2007 9 1 November 2007 13 3 Spanish Supercomputing Network (RES)
MareNostrum Processors: 10240 PowerPC 970 2.3 GHz Memory: 20 Tbytes Disc: 280 + 90 Tbytes Networks: Myrinet, Gigabit, 10/100 Operating System: Linux
CeSViMa Processors: 2408 PowerPC 970 2.2 GHz Memory: 4.7 Tbytes Disc: 63 + 47 Tbytes Networks: Myrinet, Gigabit, 10/100 Operating System: Linux
IAC, UMA, UNICAN, UNIZAR, UV Processors: 512 PowerPC 970 2.2 GHz Memory: 1 Tbyte Disc: 14 + 10 Tbytes Networks: Myrinet, Gigabit, 10/100 Operating System: Linux
The BSC-IBM MareIncognito project
10 Petaflop research project (2010)
Port/develop applications to reduce time-to-production once installed Application Fine-grain development programming an tuning models Programming models
Tools for application Performance Model and Load analysis and prototype balancing development Prediction and to support previous Tools evaluations
Interconnect Processor Evaluate node architecture and node
Evaluate interconnect options Which countries joined PRACE ir1 tier
GENCI ir0 tier
Principal Partners
General Partners
Earth Science Research at the Barcelona Supercomputing Center CALIOPE: Air Quality Forecasting System
http://salam.upc.es/caliope
Enhancing spatial resolution – Precipitation towards a new generation air quality modeling system
Resolution improved to 12 km for all Europe, 4 km for the Iberian peninsula, and 1 km for hot spot regions within MareNostrum Supercomputer
High-resolution modeling
Water vapour HERMES2004: High-Elective Resolution Modelling Emission System (Baldasano et al 2008)
Database information management
Biogenic emissions On road, ships and Industry and electric Residential and aircrafts traffic generation commercial Land use map Emission factors and Roads (urban streets, roads, Location (point/area) Density population map biomass etc.), LTO cycles and ships routes Production / electric Combustibles consume Meteorological Hourly, weekly and monthly generation Emission factors information (air temperature, profiles Emission factors – energetic Solvent use solar radiation) Distribution of the vehicles intensity Waste generation Velocity by road type Chimney height / diameter Meteorological information (air temperature) Type of combustibles
Emission computation and speciation
BVOCs NOx, VOCs, CO, SO2, particulate NOx, VOCs, CO, SO2, particulate NOx, VOCs, CO, SO2, particulate matter, CO2, CH4 and N2O matter, CO2, CH4 and N2O matter, CO2, CH4 and N2O
Speciation
Management of the emission information Management of emission data for air quality modelling
Hourly aggregation of archives (4x4 km, 1x1 km grid size) Visualization and analysis of the results Height desaggregation (vertical layers) Management information tool Database resolution (1 - 4 km grid size) Generation of netCDF emission files
HERMES2004: Emissions for Madrid, 1 km, Primary Organic Aerosol (POA)
Madrid
Barcelona
HERMES2004: Emissions for Catalonia, 1 km, Paraffin (PAR) Surface tropospheric ozone, 18-21 Jun 2007
Surface level PM2.5 sulfates, 19-21 Jun 2007 http://www.bsc.es/projects/earthscience/AirQuality/sub_fc_val_google.php
1. Operational 2. Operational 3. Operational air quality meteorological meteorological meteorological meteorological validation: OMI NO2 validation: MSG validation: RADAR tropospheric column Computational requirements
Workflow of a 48 h high-resolution simulation of Europe at 12 km and Spain at 4 km
Meteorology
Emissions
Chemistry
IMPACT OF THE EXTRATROPICAL STORM DELTA OVER THE CANARY ISLANDS ON 28-30 NOVEMBER 2005: SEVERE WINDSTORM EVENT
On 28-29 November 2005 an extratropical storm affected Canary Islands causing significant damage related to high sustained wind and intense gusts over some of the islands. Delta was the 26th tropical storm of the 2005 Atlantic hurricane season.
Station name 10 m wind speed 10 m wind gust Time UTC (m/s) (m/s) Hierro airport 18.7 32.7 18:00 Tazacorte 8.5 16.5 12:30 La Palma airport 28.9 46 19:00 La Gomera 13.3 25.5 16:00 Tenerife/Los Rodeos 8.3 19.4 16:00 Sta. Cruz de Tenerife 15.9 36.6 21:00 Tenerife/Sur 24.1 37.1 21:40 Izaña 49.9 60.6 20:00
The aim of this study is to describe the local effects of the flow that were observed over Canary Islands during the travel of the Delta storm near the region using high-resolution mesoscale meteorological simulations. The role of the complex topography of the Canary Islands will be discussed with an experimental simulation without the topography of the islands. Canary Islands archipelago
The Canary Islands are located in the middle-east of the Atlantic Ocean in front of the southern coast of Morocco. They are the most distant archipelago of Spain constituted of seven islands of volcanic origin with very complex topography. La Palma, which is a very steep island, and Tenerife, the largest island (1929 km2) with the highest peak (3718 m a.s.l.) of Spain (Teide volcano), were the most affected for the synoptic situation of Delta storm.
La Palma Lanzarote La Tenerife Palma Tenerif La e Fuerteventura GomerLa Gomera a Gran El Gran Canaria El Hierro Canari Hierro a
Methods
Weather Research and Forecasting (WRF) Model ARW dynamics solver Microphysics: single-moment 3-class scheme Cumulus: Kain-Fritsch PBL: Yonsei University PBL scheme LSM: Rapid Updated Cycle LSM Radiation: RRTM for LW, Dudhia scheme for SW IC & BC: ECMWF IFS-0.25º forecast, BC every 3h. Conducted numerical simulations: Base case run: three nested high-resolution domains 9, 3, 1 km Experimental run without Canary Islands topography 00 UTC 28 November 2005 – 00 UTC 30 November 2005 Domains: D1: 300 x 230 grid points at 9 km D2: 340 x 226 grid points at 3 km D3: 337 x 292 grid points at 1 km 40 sigma vertical levels; model top at 50 hPa Vertical structure: Tenerife
Under these conditions an internal gravity wave develops aloft Tenerife Island and La Palma as is shown in the cross sections. The wind speeds at the lee of the mountain intensifies and the downslope flow enhances. A well defined jet flow in the lee of the mountain presents a wind-speed core of 39.34 m/s at 400 m a.g.l. This jet flow impacts at the surface with wind speeds of 30-32 m/s. The vertical structure of the flow provokes important gradients in the wind field, surface regions on the lee side of the mountain separated by no more than 5 km present wind speed differences of 30 m/s. At 24 UTC the windstorm has extended downslope and its jet core presents a maximum wind speed of 45.14 m/s at 100 m a.g.l.
B
00 UTC 29 Nov 12 UTC 28 Nov 18 UTC 28 Nov 00 UTC 29 Nov MIDERAL DUST
Mineral Dust Current operational domains of simulation http://www.bsc.es/projects/earthscience/DREAM/
North Africa and Mediterranean East-Asia - 1/3 x 1/3 degree resolution (in the - 1/2 x 1/2 degree resolution 24 vertical rotated semi-staggered Arakawa E grid) levels - 24 vertical levels - 72 - hour daily forecasts - 72 - hour daily forecasts - Initialized at 00UTC - Initialized at 12UTC
Operational forecast products
SDS WS Operational products http://www.bsc.es/projects/earthscience/DREAM/ Model predictions (72-h): Horizontal distribution Z PM2.5, PM10, TSP at surface and height Z Total column mass (dust load) Z Dust aerosol optical depth Z Wet, dry, total deposition Z Visibility (soon available) Z Meteorological variables Vertical distribution Z Cross sections Z Fixed point/time profiles Fixed point (selected sites/cities) Z Dustgrams Z Meteograms
Request-only basis: Numerical data Climatology Dust forecast and validation system
Model has shown very good agreement with observations in a number of studies of single events (e.g., Ansmann et al., 2003, Papayannis et al., 2005; Pérez et al., 2006a;b; Jiménez et al, 2006 ….) Lidars - EARLINET AERONET - ONLINE Meteosat Second Generation SeaWIFS
Current users of the system: • Scientific (aerosols, ocean, health, …) • Experimental campaigns (TROMPETA, SAMUM, …) • Observational Networks: Earlinet (European Lidar Network), AERONET and in-situ observations • Satellite community • Spanish administration: alert system
Atlantic Dust Simulations 1958-2006
- Resolution: 1/3º x 1/3º horizontal and 24 levels - Time step: 120 s - Vegetation cover and soil properties constant
- Computing time single processor for 24 h simulation: ~ 70 min - Computing time single processor for 48 years: ~ 851 days (~ 28 months) - MareNostrum ~ 7 days Hurrican ISAAC – Saharan Dust
New model development: Global / Regional ESMF/NMM-b (2007-2010)
Implementation of aerosol and chemistry Eta model ESMF/NMM-b model module into the most recent parallelized and non-hydrostatic version of the NMM/ NCEP Regional model Global/Regional model model, with application either as a limited Hydrostatic Non-hydrostatic area or global model Eta coordinate Sigma coordinate b Increasing resolution Arakawa E-grid Arakawa B-grid b Global model domain b High resolution dust forecast Convection Microphysical parameterization convection
•NEW GENERATION OF EARTH SYSTEM MODELLING FRAMEWORK (ESMF) IN MARENOSTRUM PARALLELIZED ENVIRONMENT
•IMPLEMENTATION OF MINERAL DUST MODULE
•IMPLEMENTATION OF FULL CHEMISTRY COMPONENT ON- LINE
•CODE IMPROVEMENT AFTER BENCHMARK AND PERFORMANCE STUDIES PRELIMINARY RESULTS WITH DUST COMPONENT Cycle 20080124_12 54h forecast Simulation set-up: Global Cycle 20080124 12 UTC 60 hours forecast 160 CPU’s 8 dust bins 769x541 64 sigma layers
Dust conc bin 4 Sigma layer 52
kg/m3
WMO Sand and Dust Storm Warning and Assessment System (SDS WAS)
To enhance the ability of participating countries to establish and improve systems for forecasting and warning to suppress the impact of Sand and Dust Storm by Establishing a coordinated global network of Sand and Dust Storm forecasting centers delivering products useful to a wide range of users in understanding and reducing the impacts of SDS
North Africa, Middle East and Europe Asia
BSC-CNS AEMET, Spain China Meteorological Administration (CMA) Xiao Ye Zhang [email protected] WMO REGIONAL CENTRES Global Circulation Model: EC-Earth (ECMWF), GISS ModelE (NASA)
Climate modeling system at BSC-CNS: Global Circulation Model Regional Climate Model
Meteorology: WRF-ARW / WRF-NMM Regional modelling system
Mineral Dust: DREAM
Chemistry Transport: CMAQ / CHIMERE
Emissions: HERMES, EMEP, IPCC
Volcanic ash forecast (Folch et al 2008) Application to the May 2008 Chaitén volcan eruption
The BSC-CNS Earth Science Department: Dir: Dr. José M. Baldasano THANKS David Carrió CONTACT María Gonçalves [email protected] Dr. Santiago gassó Dra. Patricia Güereca Karsten Haustein Dr. Pedro Jiménez Dr. Oriol Jorba Eugeni López Thomas Loridan Simone Marras Albert Soret María Teresa Pay Dr. Carlos Pérez Ángel Rincón