Centre de Calcul de l’Institut National de Physique Nucléaire et de Physique des Particules
Astroparticles at CC-IN2P3 Rachid Lemrani, June 11th, 2014 Astroparticle experiments at CC-IN2P3
running soon later
Astroparticles Gamma HESS-II CTA Extreme energies FERMI Origin Composition Neutrino ANTARES KM3NET Anti-matter Dark Matter Gravity Cosmic rays AUGER JEM-EUSO AMS02
Gravitational Waves VIRGO Adv VIRGO ET
Cosmology Dark Matter Dark Matter EDELWEISS EURECA Dark Energy SuperNovae,WL,... SNLS / SNF LSST / EUCLID Parameters CMB PLANCK
Neutrinos : Majorana/Mass ν-less double β Decay NEMO ! SuperNEMO Mixing Neutrino Oscillation OPERA ! ! D-CHOOZ ! ! T2K ! Proton decay LAGUNA
Most computing intensive in the future : CTA, LSST, Euclid
Astroparticle experiments at CC-IN2P3, June 11th 2014 2 Planck
Planck publication : 21st march 2013
~ 5 MHS05.hour / month
- CC-IN2P3 provided increased amount of computing before the publication Planck
Astroparticle experiments at CC-IN2P3, June 11th 2014 3 Astroparticle experiments at CC-IN2P3
Auger
AMS
HESS Supernovae
ANTARES VIRGO
Planck Fermi
Opera Double-Chooz Nemo Edelweiss
Astroparticle experiments at CC-IN2P3, June 11th 2014 4 Astroparticle experiments at CC-IN2P3
DOUBLE-CHOOZ SPACE
AMS
FERMI EDELWEISS PLANCK CC-IN2P3 NEMO EUCLID
VIRGO
ANTARES OPERA
T2K
SNLS/SNF CTA ?
LSST HESS
AUGER
Astroparticle experiments at CC-IN2P3, June 11th 2014 5 Types of data
! Events : gammas, neutrinos, cosmic rays
! 1D signals : gravitational waves, CMB
! 2D Images : Supernovae
! run conditions : calibration, monitoring
Astroparticle experiments at CC-IN2P3, June 11th 2014 6 Types of storage
! Mass storage on tapes (HPSS) : large files
! Data analysis on Semi-Permanent Storage (GPFS) : SuperNovae, Planck ...
! Disks : gravitational waves, ...
! Shared files on AFS : programs, ...
! Databases : run conditions, parameters
Astroparticle experiments at CC-IN2P3, June 11th 2014 7 Data management and access
! Management with IRODS : data transfer, heterogeneous support : cartridges, disks, databases. Unix like commands to copy, list, search for files in a simple arborescence !
! Access with XROOTD : well adapted to root files (other formats also possible) interfaced at CC-IN2P3 with HPSS
Astroparticle experiments at CC-IN2P3, June 11th 2014 8 Computing types
! Parallel processing MPI : CMB ! Signal processing : GW, CMB (1D → 2D maps) ! Images processing: Supernovae ! Events processing: reconstruction, analyses, ... ! Monte-Carlo simulations : - Atmospheric showers (auger, cta) - Propagation of light in the sea (antares) - Backgrounds for rare events detection (edelweiss, supernemo) - ... ! Interactive : programs tuning before mass computing
Astroparticle experiments at CC-IN2P3, June 11th 2014 9 Future
10 Future storage increased
Increase expected by experiments
Cumulated Disk TB Cumulated Tape TB 25000 40000
20000 30000 15000 20000 10000 10000 5000
0 0 2014 2015 2016 2017 2018 2013 2014 2015 2016 2017 2018
LCG AMS DCHOOZ AUGER VIRGO ANTARES HESS-II EUCLID LSST others
Astroparticle experiments at CC-IN2P3, June 11th 2014 11 CTA
! - 2016 to 2020 ??? deployment of the array -> Increasing data flow
- From 2020 : of order 1 to 10 PB/year
Astroparticle experiments at CC-IN2P3, June 11th 2014 12 LSST : Large Synoptic Survey Telescope
- Telescope ready for system Integration and testing : July 4 2019 - Camera ready : May 15, 2020 - Commissioning phase will be 24-36 months - 6 months minimum with the Commissioning camera - 18 months with camera for I&T and science verification ! CC-IN2P3 : • half of Data Release Production (other half by NCSA / Urbana Illinois) • will host all the processed data ! Challenges : • data volume 100s PB in 2030 ? • analysis type : images • scale gap wrt SNLS / SN Factory • Huge database : 10s PB of objects, sources, ...
Astroparticle experiments at CC-IN2P3, June 11th 2014 13 LSST - DATA CHALLENGE - Summer 2013
DC-2013 : First major DC involving both CC-IN2P3 and NCSA ! Using DATA from SDSS : SDSS DR7 survey in Stripe 82 : ~5 TB for CC-IN2P3: 298 runs and nearly 4 million images in 5 passbands ! CC-IN2P3 : - 1 millions CPU hours (HS06) - 40 000 jobs - Up to 700 batch slots - One dedicated interactive VM for debug, file transfer, ... - Tests of job submission using DIRAC (on z filter) - Up to 100 TB on sps (GPFS) / 20 million files - Data transfers using IRODS : 100 TB - MySQL server for produced catalog of objects ! Very challenging with only 5 TB of Raw data
Astroparticle experiments at CC-IN2P3, June 11th 2014 14 LSST - QSERV : Distributed LSST database Developped by SLAC & Caltech, contributions by LPC/IN2P3 Clermont Will host 34 PB with 9000 billons entries
Overlapping partitions of the sky
request results
CC-IN2P3 : Test of QSERV platform with 310 nodes (in 2013)
Very usefull uncovering unexpected issues : QSERV didn’t scale with more than 250 nodes
Astroparticle experiments at CC-IN2P3, June 11th 2014 15 EUCLID
■ Space mission 7 years, launch in 2020 - Dark Energy / Dark Matter, Gravity, Large Scale Structures ■ 2026 : ~ total 150 PB ? ■ 8 Science Data Centers ! ! ! ! ! ! ■ CC-IN2P3 : 30 % of total Processing and Storage !
Astroparticle experiments at CC-IN2P3, June 11th 2014 16 Advanced Virgo
Virgo shutdown Advanced Virgo : starting end 2015 ? 800 TB / year at CC-IN2P3 from 2019 ? ! - Computing : mainly at CC-IN2P3 and CNAF ! - Preparation for computing model for adv. virgo : ! challenges : type/amount of computing (GPU), heterogeneous clusters !
Virgo
Astroparticle experiments at CC-IN2P3, June 11th 2014 17 Prospects KM3NET high energy neutrinos
CTA : gamma rays AUGER upgrade/extension JEM-EUSO : extreme energies
EURECA : Dark Matter SUPERNEMO LAGUNA proton decay, neutrino v-less double beta decay
advVIRGO /ET Gravitational waves
LISA gravitational waves Euclid, LSST, Chile dark energy dark energy
Astroparticle experiments at CC-IN2P3, June 11th 2014 18 Backup Slides
19 CPU Consumption in 2012
2012 [HS06 Hrs] antares 22 637 176 pauger 17 549 325 hess 17 074 751 HESS dchooz 14 902 802 AUGER planck 9 255 537 ams 6 246 575 glast 6 051 319 16 % 15 % nemo 4 179 202 km3net 4 037 867 virgo 3 079 917 DOUBLE- t2k 1 175 106 CHOOZ nusol 965 334 14 % snovae 831 608 21 % trend 642 864 ANTARES lsst 536 866 cta 536 303 8 % xmm 247 230 integral 163 779 6 % sdss 117 093 5 % PLANCK 4 % opera 105 090 nucifer 89 243 jemeuso 62 015 AMS baoradio 7 686 NEMO edelweis 5 588 FERMI Total 110 253 047
Astroparticle experiments at CC-IN2P3, June 11th 2014 20 Storage in 2012
- Tapes (HPSS)
Mass Storage (TB) ams 885 VIRGO ANTARES virgo 819 819 TB antares 591 591 TB dchooz 587 hess 462 pauger 332 210 nemo DOUBLE- 153 snovae AMS baoradio 88 CHOOZ 885 TB cta 81 587 TB km3net 40 edelweis 31 t2k 29 sdss 27 planck 21 HESS 20 trend 462 TB NEMO AUGER 210 TB 332 TB
- High performance disks (GPFS) : PLANCK : ~ 150TB SNLS : ~ 140 TB SNF : ~ 65TB
Astroparticle experiments at CC-IN2P3, June 11th 2014 21