RES updates, resources and Access European HPC ecosystem
Sergi Girona RES Coordinator RES: HPC Services for Spain
•The RES was created in 2006. •It is coordinated by the Barcelona Supercomputing Center (BSC-CNS). •It forms part of the Spanish “Map of Unique Scientific and Technical Infrastructures” (ICTS). RES: HPC Services for Spain
RES is made up of 12 institutions and 13 supercomputers.
BSC MareNostrum CESGA FinisTerrae CSUC Pirineus & Canigo BSC MinoTauro UV Tirant UC Altamira UPM Magerit CénitS Lusitania & SandyBridge UMA Picaso IAC La Palma UZ CaesarAugusta SCAYLE Caléndula UAM Cibeles 1 10 100 1000 TFlop/s (logarithmic scale) RES: HPC Services for Spain • Objective: coordinate and manage high performance computing services to promote the progress of excellent science and innovation in Spain. • It offers HPC services for non-profit, open R&D purposes. • Since 2006, it has granted more than 1,000 Million CPU hours to 2,473 research activities. Research areas
Hours granted per area
23% AECT 30% Mathematics, physics Astronomy, space and engineering and earth sciences BCV FI QCM 19% 28%
Life and health Chemistry and sciences materials sciences RES supercomputers
BSC (MareNostrum 4) 165888 cores, 11400 Tflops Main processors: Intel(R) Xeon(R) Platinum 8160 Memory: 390 TB Disk: 14 PB
UPM (Magerit II) 3920 cores, 103 Tflops Main processors : IBM Power7 3.3 GHz Memory: 7840 GB Disk: 1728 TB
UMA (Picasso) 4016 cores, 84Tflops Main processors: Intel SandyBridge-EP E5-2670 Memory: 22400 GB Disk: 720 TB
UV (Tirant 3) 5376 cores, 111,8 Tflops Main processors: Intel SandyBridge-EP E5-2670 Memory: 10752 GB Disk: 14 + 10 TB
CSUC (Pirineus) 2784 cores, 283,66 Tflops Main processors: Intel(R) Xeon(R) Platinum 8160 Memory: 12000 GB Disk: 200 TB
CSUC (Canigo) 384 cores, 33,2 Tflops Main processors: Intel(R) Xeon(R) Platinum 8160 Memory: 9000 GB Disk: 200 TB RES supercomputers CénitS (Lusitania 2) 800 cores, 33,2 Tflops Main processors Intel Xeon E5-2660v3, 2.6GHz Memory: 10 GB Disk: 328 TB
CénitS (SandyBridge) 2688 cores, 56 Tflops Main processors Intel Sandybridge Xeon Memory: 5376 GB Disk: 328 TB
BSC (MinoTauro) 624 cores, 251 Tflops Main processor: 39x 2 Intel Xeon E5-2630 v3 Memory: 20 TB Disk: 14PB (shared with MN4)
CESGA (FinisTerrae 2) 7712 cores, 328,3Tflops Main processor: Intel Xeon E5-2680v3 Memory: 40 TB Disk: 960 TB
UC (Altamira 2+) 5120 cores, 105 Tflops Main processor: Intel SandyBridge Memory: 15,4 TB Disk: 2PB
UZ (Caesaraugusta) 2014 cores, 80.5 Tflops Main processor: Intel E5-2680v3, 2.5GHz Memory: 5400 GB RAM memory Disk: 219TB RES supercomputers SCAYLE (Caléndula) 2432 cores, 50,6 Tflops Main processor: Intel SandyBridge Xeon Memory: 4864 GB Disk: 600 TB
UAM (Cibeles) 368 cores, 14,1 Tflops Main processor: Intel Xeon E5-2630 v3, 2.40GHz Memory: 896 GB Disk: 80 TB
UAM (SandyBridge) – coming soon – 2688 cores, 56Tflops Main processor: Intel SandyBridge Xeon, 2.60GHz Memory: 5376 GB Disk: 80 TB
IAC (LaPalma) 4032 cores, 83,85 Tflops Main processor: Intel SandyBrigde Memory: 8064 GB Disk: 60 TB Resources granted: CPU hours
400.000
350.000
300.000
140mh available 250.000
200.000 Hours x 1000 x Hours 150.000
100.000
50.000
0 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017
Requested hours Awarded hours (A+B) How to apply?
• RES resources are open to researchers and spin-offs:
o Computing resources: CPU hours and storage o Technical support: application analysis, porting of applications, search for the best algorithm… to improve performance and ensure the most effective use of HPC resources. o Free of cost at the point of usage
Next deadline: • Three open competitive calls per year. January 2019
Deadline for Period Starting date applications P1 January 1st March P2 May 1st July P3 September 1st November How to apply?
RES intranet: https://www.bsc.es/res-intranet
• Researchers present a proposal which includes research project description, technical requirements and research group experience.
• Accepted proposals have access to RES supercomputers for 4 months.
• Granted time can be: hours with priority (hours A) or without priority (hours B) Proposal evaluation
Formal Technical evaluation evaluation Submit
Access committee Scientific experts panel
Final report of accepted activities Activity length
• Accepted proposals have access to RES supercomputers for 4 months. If your activity needs more time to be properly developed, you can ask for a continuation activity:
New activity Report dissemination (4 months) information
Continuation activity Report dissemination (4 months) information
• The application form is simplified • Are preferably allocated to the same machine Continuation activities • In the evaluation, one reviewer is kept from the previous activity and the second reviewer changes RES Users’ Committee
• CURES aims to provide advice and feedback to RES coordinators:
o Promotes optimal use of high performance computing facilities
o Shares information about users’ experiences
o Voices user concerns
• You can contact CURES through RES intranet: Which is/could be the main impediment to apply to RES resources?
Ø I don't know how to write a strong application Ø I'm not sure if I can apply Ø Lack of HPC expertise in my research group Ø Too much paperwork Ø Not enough resources for my project Tips to write a strong proposal
• Read carefully all the protocols, guides and FAQs in:
https://www.res.es/en/access-to-res
• Project description section: highlight the importance of your project, not only from the scientific point of view but also from the return to society.
Ø Why do you think that your project should deserve the resources requested? Tips to write a strong proposal
• Activity description section: specify clearly why do you need supercomputing resources. Write as accurate as possible the flowchart of the simulations. Indicate that you have your own human resources in your group to run and process the output of all the simulations you propose.
Ø Why do you need to carry out the simulations in the selected machine?
Ø Is the amount of computing resources requested adjusted to your needs and properly justified?
• Doubts about software/HPC resources: ask support team!
Ø Are your jobs adequate for parallel computing?
[email protected] Which is/could be the main impediment to apply to RES resources?
Ø I don't know how to write a strong application Ø I'm not sure if I can apply Ø Lack of HPC expertise in my research group Ø Too much paperwork Ø Not enough resources for my project Who can apply?
• RES resources are aimed at open R+D+I activities:
o Researchers from academia and public R&D institutions o Spin-offs during their first 3 years from its creation o Collaboration projects between private companies and research groups from academia or public institutions o Open to international applicants, but we recommend the collaboration with researchers from Spanish institutions Which is/could be the main impediment to apply to RES resources?
Ø I don't know how to write a strong application Ø I'm not sure if I can apply Ø Lack of HPC expertise in my research group Ø Too much paperwork Ø Not enough resources for my project RES events: technical training
These workshops are organized by the RES nodes and aim to provide the knowledge and skills needed to use and manage the supercomputing facilities.
• Check the agenda in RES website: https://www.res.es/en/events?event_type=technical_training
• PATC courses in BSC (PRACE Advanced Training Center): https://www.bsc.es/education/training/patc-courses RES events: networking opportunities Scientific seminars
The RES promotes scientific seminars which address supercomputing technology applications in specific scientific areas. These events are mainly organized by RES users and are open to the entire research community.
In 2017:
ü 5 scientific seminars ü More than 300 attendees
Sep Next Generation Sequencing and Supercomputing: life as a couple 27 CBMSO-UAM (Madrid)
Agenda 2018: www.res.es/en/events RES events: networking opportunities
RES Users’ Meeting: 20 September 2018 - Valencia
The agenda includes: • Information about RES and the European HPC ecosystem • Plenary session: Research Open Data • Parallel scientific sessions • Poster session • Networking opportunities • Evening social event www.res.es/users-conference-2018 Funded by the EC: 2017 - 2021
ü Mobility grants for researchers using HPC resources ü Short stays to visit scientific hosts (3 weeks – 3 months) ü Funds for travel and living allowance ü Access to European HPC facilities
Next deadline: 20 September
http://www.hpc-europa.eu/ Which is/could be the main impediment to apply to RES resources?
Ø I don't know how to write a strong application Ø I'm not sure if I can apply Ø Lack of HPC expertise in my research group Ø Too much paperwork Ø Not enough resources for my project RES forms
In the RES we try to keep the administrative procedures short and simple for researchers:
• New activity application form: 10 pages on average
• Continuation activity application form: simplified
• Dissemination form: 3 pages on average o Brief description of results (1-2 paragraphs) o Publications o Figures / pictures o Optional: patents, PhD students…
• Intermediate reports: 1-2 sentences (“Everything is ok”)
• Resubmission of non-accepted activities: one click Which is/could be the main impediment to apply to RES resources?
Ø I don't know how to write a strong application Ø I'm not sure if I can apply Ø Lack of HPC expertise in my research group Ø Too much paperwork Ø Not enough resources for my project PRACE HPC Access
• Call for Proposals for Project Access: Next deadline: - 12, 24 or 36-month projects 30 October - Minimum request: 30 million core hours
• Call for Proposals for PRACE Preparatory Access:
- From 2 to 12 month projects
http://www.prace-ri.eu/ EuroHPC Distributed Supercomputing Infrastructure
24 members, including 5 Hosting Members
(Switzerland, France, Germany, Hazel Hen Italy and Spain) JUQUEEN
SuperMUC 524 scientific projects enabled Curie
70 PFlops/s of peak performance Piz Daint on 7 world-class systems Marconi
>10.000 people trained by 6 PRACE Advanced Training Centers and MareNostrum others events
Access prace-ri.eu/hpc_acces Top500 (June 2018), European ranking
Cores & Rmax Rpeak Europe Rank Name Computer Site Country Accel. [PFlop/s] [PFlop/s] Cray XC50, Xeon E5-2690v3 12C 361760 1 6 Piz Daint 2.6GHz, Aries interconnect , NVIDIA CSCS Switzerland 19,59 25,33 297920 Tesla P100 Proliant DL380 Gen10, Xeon Platinum 253600 2 13 HPC4 8160 24C 2.1GHz, Mellanox ENI Italy 12,21 18,62 177520 InfiniBand EDR, NVIDIA Tesla P100 Bull Sequana X1000, Intel Xeon Phi 3 14 Tera-1000-2 CEA France 561408 11,97 23,40 7250 68C 1.4GHz, Bull BXI 1.2 CINECA Cluster, Lenovo SD530/S720AP, Intel Xeon Phi 7250 4 18 Marconi Intel Xeon Phi CINECA Italy 312936 8,41 16,21 68C 1.4GHz/Platinum 8160, Intel Omni-Path Cray XC40, Xeon E5-2695v4 18C 5 20 UKMET United Kingdom 241920 7,04 8,13 2.1GHz, Aries interconnect Lenovo SD530, Xeon Platinum 8160 6 22 MareNostrum BSC Spain 153216 6,47 10,30 24C 2.1GHz, Intel Omni-Path Bull Sequana X1000, Xeon Platinum 8168 24C 2.7GHz, Mellanox EDR 7 23 JUWELS Module 1 Juelich Germany 114480 6,18 9,89 InfiniBand/ParTec ParaStation ClusterSuite Cray XC40, Xeon E5-2680v3 12C 8 27 Hazel Hen HLRS Germany 185088 5,64 7,40 2.5GHz, Aries interconnect Intel Compute Module HNS2600BP, Max-Planck- 9 28 COBRA Xeon Gold 6148 20C 2.4GHz, Intel Gesellschaft Germany 127520 5,61 9,79 Omni-Path MPI/IPP SGI ICE X, Xeon Xeon E5-2670/ E5- 10 30 Pangea Total France 220800 5,28 6,71 2680v3 12C 2.5GHz, Infiniband FDR Top10 industrial machines (June 2018)
Cores & Rmax Rpeak Rank Name Computer Site Country Acc. [PFlop/s] [PFlop/s] Proliant DL380 Gen10, Xeon Platinum 8160 24C 2.1GHz, Mellanox 253600 13 HPC4 Eni S.p.A. Italy 12,21 18,62 InfiniBand EDR, NVIDIA Tesla P100 177520 SGI ICE X, Xeon Xeon E5-2670/ E5-2680v3 12C 2.5GHz, Infiniband Total Exploration 30 Pangea France 220800 5,28 6,71 FDR Production Petroleum Geo- 35 Abel Cray XC30, Xeon E5-2698v3 16C 2.3GHz, Aries interconnect United States 145920 4,04 5,37 Services NVIDIA DGX-1/Relion 2904GT, Xeon E5-2698v4 20C 2.2GHz/ E5- 60512 45 Facebook United States 3,31 4,90 2650v4, Infiniband EDR, NVIDIA Tesla P100/Quadro GP100 55552 NVIDIA DGX-1, Xeon E5-2698v4 20C 2.2GHz, Infiniband EDR, NVIDIA 60512 46 DGX Saturn V NVIDIA Corporation United States 3,31 4,90 Tesla P100 55552 iDataPlex DX360M4, Intel Xeon E5-2680v2 10C 2.8GHz, Infiniband 72000 49 HPC2 Eni S.p.A. Italy 3,19 4,61 FDR, NVIDIA K20x 42000 Lenovo NeXtScale nx360M5, Xeon E5-2697v4 18C 2.3GHz, Infiniband Energy Company 66000 65 HPC3 Italy 2,59 3,80 EDR, NVIDIA Tesla K80 (A) 39000 PowerEdge R440/C6320p, Intel Xeon Phi 7250/Xeon Gold 6130 16C 78 Makman-3 Saudi Aramco Saudi Arabia 53300 2,32 3,58 2.1GHz, Intel Omni-Path Inspur TS10000, Xeon Gold 6130 16C 2.1GHz, NVIDIA Tesla V100, 55104 79 Internet Service P China 2,29 4,89 25G Ethernet 52480
80 Makman-2 Dell PowerEdge R630, Xeon E5-2680v3 12C 2.5GHz, Infiniband QDR Saudi Aramco Saudi Arabia 76032 2,25 3,04 Signatory European Countries Status of Signatory Countries Signatory of Status EuroHPC mission and objectives
• To develop, deploy, extend and maintain in the Union an integrated world-class supercomputing and data infrastructure and to develop and support a highly competitive and innovative High-Performance Computing ecosystem • Provide European scientists, industry and the public sector with the latest HPC and data infrastructure and support the development of its technologies and applications across a wide range of fields • Support an ambitious research and innovation agenda to develop and maintain in the Union world-class High Performance & Skills Computing ecosystem, exascale and beyond, covering all Operations
scientific and industrial value chain segments, including low- & Infrastructure power processor and middleware technologies, algorithms and Applications R&I, code design, applications and systems, services and engineering, interconnections, knowhow and skills, for the next generation HPC Ecosystem supercomputing era Infrastructure and Operation
• Two exascale systems • Possibly one on 2022 and a second one in 2023 • At least one with European competitive technologies • Expected budget on TCO: 500 m€ each • Two pre-exascale systems • In operation in January 2021 • Expected budget on TCO: 240m€ each • At least 2 petascale systems • In operation in January 2021 • Budget: TBD • Next steps • Identify the hosting sites • User requirements to prepare RFP EuroHPC Roadmap
HPC and Accelerator Chips & Systems Gen 1 Gen 2 2018 2019 2020 2021 2022 2023 2024 2025 EuroHPC JU
Software Software sw stack Apps Applications
R&I Co-Design App&Software
EsD Intg. Integration
Exascale technologies integration Procurement 2 Pre-Exa 2 Exascale (1 EU) 2 Peta Visit our website: www.res.es
Subscribe to our newsletter
Contact us! Follow us in Twitter: @RES_HPC
[email protected] [email protected] THANK YOU!