ASPERA Meeting, Paris 8 Feb 2008 Structure
Total Page:16
File Type:pdf, Size:1020Kb
MARENOSTRUM at BSC (Barcelona SuperComputing Center) & RES (The Spanish Supercomputing Network) Prepared by Sergi Girona ASPERA Director Operaciones BSC-CNS Presented by Jesus Marco Meeting, Paris RES Universidad de Cantabria Centro Nacional de Supercomputación ● BSC-CNS mission: ● Research, develop & manage technology to promote science advance ● BSC-CNS objectives: ● R&D in: ● Computational Sciences ● Life Sciences ● Earth Sciences. ● Supercomputing Support to external research. ● BSC-CNS is a Consortium with the following partners: ● Spanish Ministry of Education & Science (MEC) – 51% ● Government of Catalonia (Generalitat, DIUE) – 37% ● Universitat Politècnica de Catalunya (UPC) – 12% ASPERA Meeting, Paris 8 Feb 2008 Structure ASPERA Meeting, Paris 8 Feb 2008 Computer Sciences Department Computer architecture: Benchmarking, analysis and prediction tools: • Highly o-o-o architectures • Tracing scalability • Hardware multithreading • Pattern and structure identification • Design space exploration for multicore • Visualization and analysis chips and Hw accelerators • Processor, memory, network, system • Transactional memory (Hw, Hw-assisted) • SIMD and vector extensions/units The Grid Programming models: • Scalability of MPI and UPC • OpenMP for multicore, SMP Future Petaflop and ccNUMA systems • DSM for clusters Large cluster • CellSs systems Grid and cluster computing: • Transactional Memory • Programming models • Embedded architectures • Resource management Small DMM • I/O for Grid Operating environments: cc-NUMA • Autonomic application servers Chip On-board SMP • Resource management for heterogeneous workloads • Coordinated scheduling and resource management • Parallel file system scalability ASPERA Meeting, Paris 8 Feb 2008 Life Sciences Department THE ENCODE THE EARTH PROJECT PROTEOME ● Genome all organisms + all metagenomes ● 1.4x106 sequences ● 32 Tb of RAM used ● 4000 processors ● 38 CPU years ● 9 Tb of output data ● G-Superscalar ● 3.5 days job! ● 5 DAYS lagtime!!! ASPERA Meeting, Paris 8 Feb 2008 Earth Sciences Department Resolution improved to 12 km for all Europe, 4 km for the Iberian peninsula, and 1 km for hot spot regions within MareNostrum Supercomputer Surface tropospheric ozone, 18-21 Jun 2007 ASPERA Meeting, Paris 8 Feb 2008 CASE Department ● Aero-acoustic coupling ● 10,000 processors; 95% parallel efficiency ASPERA Meeting, Paris 8 Feb 2008 A unique place ………… ASPERA Meeting, Paris 8 Feb 2008 MareNostrum “It is not the most powerful supercomputer in the world, but it is the most beautiful” (Fortune, Sept. 2006) ASPERA Meeting, Paris 8 Feb 2008 MareNostrum ● Processors ● 2560 JS21, 2.3GHz ● 4 cores per board ● 8 Gbytes RAM ● 36 Gbytes SAS disk ● Networking ● Myrinet ● 2 Spine 1280 ● 10 Clos256 Blade centers Storage servers Gigabit switch ● 2560 Myrinet cards Myrinet racks Operations rack 10/100 switches ● Gigabit ● 10/100 Ethernet 94,20 TFlops ● Disk 10240 processors ● GPFS over 20 nodes 20 TBytes memory 280 + 90 TBytes disk ● OS ● Linux ASPERA Meeting, Paris 8 Feb 2008 Processor, Blade 4 Processors 8 Gbytes Memory Daughter card Myrinet 36 Gbytes Disk ASPERA Meeting, Paris 8 Feb 2008 Blades, blade center and blade center racks Blade Center • 14 blades per chassis (7U) • 56 processors JS21 Processor Blade • 112GB memory • 2-way 2.3 GHz Power PC 970 MP • Gigabit ethernet switch • 8GB memory (512KB L2 cache) • Local SAS drive (36 GB) • 2x1Gb Ethernet on board • Myrinet daughter card 6 chassis in a rack (42U) • 336 processors • 672GB memory ASPERA Meeting, Paris 8 Feb 2008 Hardware/Myrinet Bandwidth: 250 Mbytes Latency: 4 micros ASPERA Meeting, Paris 8 Feb 2008 Operation: money saving ASPERA Meeting, Paris 8 Feb 2008 MareNostrum ● MareNostrum Evolution: World European List rank rank November 2004 4 1 June 2005 5 1 November 2005 8 1 June 2006 11 3 November 2006 5 1 June 2007 9 1 November 2007 13 3 ASPERA Meeting, Paris 8 Feb 2008 MareNostrum, a live Mare ● MareNostrum 20042006 ● 481210240 PowerPCPowerPC 970 970 cores cores ● 24062560 JS20JS21 2.22.3 GHz ● 1020 TB of Memory ● 48 GB per node ● 270380 TB Storage Capacity ● 3 networks ● Myrinet ● Gigabit ● 10/100 Ethernet ● Operating System ● Linux 2.6 (SuSE) ASPERA Meeting, Paris 8 Feb 2008 Spanish Supercomputing Network / Red Española de Supercomputación RES MareNostrum Processor: 10240 PowerPC 970 2.3 GHz Memory: 20 TBytes Disk: 280 + 90 TBytes Network: Myrinet, Gigabit, 10/100 System: Linux UPM Processor: 2408 PowerPC 970 2.2 GHz Memory: 4.7 TBytes Disk: 63 + 47 TBytes Network: Myrinet, Gigabit, 10/100 System: Linux IAC, UMA, UC, UZ, UV Process: 512 PowerPC 970 2.2 GHz Memory: 1 TByte Disk: 14 + 10 TBytes Network: Myrinet, Gigabit, 10/100 System: Linux ASPERA Meeting, Paris 8 Feb 2008 Magerit Altamira Picasso CesarAugusta LaPalma Tirant ASPERA Meeting, Paris 8 Feb 2008 Red Española de Supercomputación ● Spanish Ministry of Education & Science (MEC) has created a distributed network of supercomputers to support the supercomputing needs of the Spanish research groups. ● Initial nodes of the network are placed at BSC-CNS, at Universidad Politécnica de Madrid (UPM), at IAC (Instituto de Astrofísica de Canarias) and at the Universities of Cantabria, Málaga, Valencia y Zaragoza. ● Access to the resources of the Red Española de Supercomputación is granted to a research group by a Single Access Committee formed by independent prestigious spanish researchers. ● BSC-CNS coordinates the joint technical management of the network. ASPERA Meeting, Paris 8 Feb 2008 Red Española de Supercomputación ● Sharing computing time ● 20% managed by the hosting institution ● 80% assigned through the Access Committee ● Single Access Committee ● 44 Spanish researchers ● 4 panels ● Astronomy, Space & Earth Sciences ● Biomedicine & Life Sciences ● Physics & Engineering ● Chemistry & Materials ● Resources assigned in four months periods ASPERA Meeting, Paris 8 Feb 2008 Access Committee QUALITY OF THE PROJECT (>30%) MPACT OF COMPUTING AT MN HPC EXPERIENCE NEED OF MN (>30%) SUITS THE ARCHITECTURE (>30%) DETAILED PROJECT EXPERTS PANEL CAN ASK FOR A PEER REVIEW THROUGH ANEP GENERAL PROJECT EXPERTS PANEL PROJECT ALREADY EVALUATED POSITIVELY SUPPORT PRIORITIZATION (WRITTEN FORM) APLICATION FORM PRIORITIZATION LISTING DECISION ACCESS BSC RESEARCHER BSC IT ANEP COMMITTEE DIRECTION (ACCEPTANCE) PEER REVIEW ACUSE DE RECIB COMPROMISO TIEMPO RESPUESTA DEFECTO DE FORMA QUALITY OF THE SCIENTIFIC PROJECT (>30%) IMPACT OF COMPUTING AT MN GROUP LEVEL HPC EXPERIENCE ASPERA Meeting, Paris 8 Feb 2008 Evolution of requested/offered hours 60 50 40 Solicitado 30 Ofertado Millones de horas de Millones 20 Tirant2007 10 CesarAugusta2007 Altamira 2007 Picaso2007 MareNostrum Magerit 2006 2007 LaPalma2007 0 2006Q2 2006Q3 2006Q4 2007P1 2007P2 2007P3 2008P1 ASPERA Meeting, Paris 8 Feb 2008 Research users at MN: ● MN has provided support to more than 300 external projects ● Distribution by scientific area: 35 30 Astronomy, Space 25 and Earth Science Biomedicine and Life 20 Science 15 Physics and Engineering Porcentaje uso 10 Chemistry and Material Science 5 0 06 07 2006 20 0 2_ 2 2 P P3_ P4 2006 P1 2007 P ASPERA Meeting, Paris 8 Feb 2008 PRACE: an Strategic European Answer tier 1 GENCI Ecosystem tier 0 Principal Partners General Partners ASPERA Meeting, Paris 8 Feb 2008 European Projects - DEISA • Consortium of leading national supercomputing centres that currently deploys and operates a persistent, production quality, distributed supercomputing environment with continental scope •Objectives: • To deploy and operate a persistent, production quality, distributed supercomputing environment with continental scope • To enable scientific discovery across a broad spectrum of science and technology. Scientific impact is the only criterion for sucess • Partners: IDRIS – CNRS, Forschungszentrum Jülich, Rechenzentrum Garching of the Max Planck Society, CINECA, CSC, SARA, European Center for Medium – Range Weather Forecasts, Leibniz computing Centre of the Bavarian Academy of Sciences and Humanities, HRLS, Barcelona Supercomputing Center. ASPERA Meeting, Paris 8 Feb 2008 Mobility Programs: HPC - Europe • Consortium of six leading High Performance Computing infrastructures and six centers of excellence: BSC - CNS (Barcelona), CINECA (Bologna), EPCC (Edinburgh), HLRS (Stuttgart), IDRIS (Paris) and SARA (Amsterdam). • Objectives: aims to provide advanced computational services in an integrated manner to the European research community working at the frontiers of science • Services: It enables researchers working in any eligible country in Europe to visit a participating research institute to carry out a collaborative visit of up to 3 months’ duration, and to gain access to some of the most powerful High Performance Computing facilities in Europe • Supported Projects: in any field of computational science, including theoretical physics, condensed matter physics, material sciences, computational fluid dynamics, astrophysics, meteo- climatology, computer science, numerical methods in engineering, bio-engineering, computational chemistry, molecular modeling, genomics and proteomics, computational biology and life sciences www.hpc-europa.org ASPERA Meeting, Paris 8 Feb 2008 What can we offer? ● Access to MareNostrum: Access Committee ● Support on usage ● Support on parallelization ● Mobility programs for doctoral candidates, postdoctoral and senior researchers ● As indicated, through HPC-Europa project (funded by EC) ● As an “Infraestructura Científica y Tecnológica Singular del Estado Español”, MEC offers support to ● Get familiar with the infrastructure ● Perform a scientific experiment ASPERA Meeting, Paris 8 Feb 2008 Research leaders with access to MareNostrum ASPERA Meeting, Paris 8 Feb 2008 Research leaders with access to MareNostrum ASPERA Meeting, Paris 8 Feb 2008 Research leaders with access to MareNostrum ASPERA Meeting, Paris 8 Feb 2008 Thanks ! http://www.bsc.es ASPERA Meeting, Paris 8 Feb 2008.