HPC Case Study

Customer Type No. of CPU Peak Perf. RIKEN (Kobe AICS) The K computer 88,128 CPUs 11.28 PFlops Australian National University (NCI) x86 Cluster (CX400) 3,592 CPUs 1.2 PFlops System operation will start in early 2013 University of Tokyo FX10 4,800 CPUs 1.1 PFlops Kyushu University x86 Cluster (CX400), 510 TFlops 3,720 CPUs System operation will start in July 2012 FX10 182 TFlops HPC Wales, UK x86 Cluster > 2,000 CPUs > 300 TFlops Japan Atomic Energy Agency x86 Cluster, FX1, SMP > 4,568 CPUs 214 TFlops Institute for Molecular Science x86 Cluster (RX300), FX10 > 420 CPUs > 140 TFlops Japan Aerospace Exploration Agency FX1, SMP > 3,392 CPUs > 135 TFlops RIKEN (Wako Lab. RICC) x86 Cluster (RX200) > 2,048 CPUs 108 TFlops NAGOYA University x86 Cluster (HX600), FX1, SMP 1,504 CPUs 60 TFlops A*STAR, Singapore x86 Cluster (BX900) 900 CPUs > 45 TFlops A Manufacturer x86 Cluster > 2,600 CPUs > 77 TFlops B Manufacturer x86 Cluster > 2,000 CPUs > 38 TFlops Type definitions: FX10=PRIMEHPC FX10, x86 Cluster=Clusters based on PRIMERGY x86 server, SMP= SPARC Enterprise SMP server 1 Copyright 2012 FUJITSU LIMITED The University of Tokyo

 Key requirements  Increasing number of users and diverseness  Software compatibility with the K computer  1.4MW of power ceiling  System overview  PRIMEHPC FX10 (4,800 nodes (50 racks)) #18 on TOP500  Peak performance: 1.13 petaflops  Linpack performance: 1.04 petaflops (91.8% efficiency)  Focusing areas: earth science, astrophysics, seismology, weather modeling, materials science, energy, biology, hydrodynamics, solid-state physics…

C計o算mノpuーtドe群 nod・インesタ,ラ Iクntテeィracブノtードive 群nodes Management servers PRIMEPHRPIMC EFXH10PC 5 0F筐体構X10 x成 50 racks (4,800 計算ノード + 300 IOノード) (4,800 compute nodes ) Job management Operation management [総理論演算性能 : 1.13PeakPFL POerPfoSr]mance: 1.13 petaflops Authentication [総主記憶容量 : 150TMBeymteo]ry capacity: 150 TB [インターコネクト : 6次I元nteメrッconnシュec/トーt: 6ラDス m] esh/torus - ”Tofu” PRIMERGY RX200S6 x 16 External connection router Local file system External Ethernet InfiniBand file system PRIMERGY RX300 S6 x 2 (MDS) network ETERNUS DX80 S2 x 150 (OST) network

Storage capacity: 1.1PB (RAID-5) Campus LAN Shared file system

PRIMERGY RX300 S6 x 8 (MDS) PRIMERGY RX300 S6 x 40 (OSS) End users ETERNUS DX80 S2 x 4 (MDT) Log-in nodes ETERNUS DX410 S2 x 80 (OST) InfiniBand Ethernet Storage capacity: 2.1PB (RAID-6) PRIMERGY RX300 S6 x 8 FibreChanneｌ

Total peak performance: 691.7 teraflops  Features: Operations beginning: July 2012  Hybrid system of Fujitsu SPARC64 and x86 cluster  Software compatibility with the K computer  Supercomputer System  PRIMEHPC FX10 (768 nodes, SPARC64 IXfx)  Peak performance: 181.6 teraflops  High-performance Server System  PRIMERGY CX400 (2,952 CPUs, New Intel Xeon E5)  Peak Performance: 510.1 teraflops

Local File System Supercomputer System High Performance ServerSystem (FEFS) PRIMEHPC FX10 (768 nodes) PRIMERGY CX400 (1,476 nodes) ETERNUS DX80 S2 181.6TFLOPS 510.1TFLOPS 345.6TB 24TB memory 184.5TB memory

InfiniBand Switch Gigabit Ethernet Switch

Shared File System (FEFS) LAN File Servers: PRIMERGY RX300 S7

4.0PB + 0.2PB Supercomputer: operation starts from July 2012 ETERNUS DX80 S2 High Performance Server System: from Sept 2012

5 Copyright 2012 FUJITSU LIMITED NCI-NF (Australia's national research computing service)  Key requirements  To improve the computational modeling capability in the research field below - Climate change - Ocean and marine - Earth system science - National water management research  Very high-energy efficiency, PUE is well under 1.20  System overview  PRIMERGY CX400 (Including CX250S1) : 3,592 nodes (50 racks) Peak performance : 1.2 Petaflops  PRIMEHPC FX10 : 96 nodes (1 rack)

C計o算mノpuーtドe群 nod・インeタsラクティブノード群 Management servers PPRRIMIMEHEPRCG FYX C10X 54000筐体構 x 50成 racks (4(C,800X250S1 計算ノー: 3ド,592+ 300 node IOノsー, ド57) ,472 cores) Management, authentication servers： PRIMERGY RX300S7 x 7 Peak Performance: 1.2 Petaflops [総理論演算性能 : 1.13PFLOPS] Memory capacity: 150 TB 総主記憶容量 [ : 150InteTrBcyonnte]ect: Full bisection bandwidth (FDR) [インターコネクト : 6次元メッシュ/トーラス] External connection router Corroboration nodes PRIMEHPC FX10 Ethernet (96 compute nodes) InfiniBand network network Peak Performance: 22.7 Tflops Memory capacity: 3 TB Campus LAN Global file system

PRIMERGY RX300S7 x 6 (MDS) PRIMERGY RX300S7 x 30 (OSS) Login nodes End users DDN EF3015 x 3 (MDT) DDN SFA12000 x 5 (OST) Login, Data mover servers： InfiniBand PRIMERGY RX300S7 x 13 Ethernet Storage capacity: 12.6PB FibreChanneｌ

Motivation and background  Position Wales at the forefront of supercomputing  Promotion of research, technology and skills  Improvement of economic development  Creation of 400+ quality jobs, 10+ new business Implementation and rollout  Distributed HPC clusters among 15 academic sites  With central hubs, tier 1 and 2 sites  Portal for transparent, easy use of resources  Rollout completed by Q1 2012

Performance & Technology Solution Design  >1400 nodes PRIMERGY BX922S2  User-focused solution to access  Intel Xeon, X5650 and X5680 distributed HPC systems from desktop  Roadmap for upgrade browser  190 TFlops aggregated peak  Multiple components integrated into a performance consistent environment with a single sign-on  Infiniband, 10 / 1 Gb Ethernet, FCS  Data accessible across the entire  Eternus DX online SAN (home FS) infrastructure, automated movement  Parallel File System (up to 10 GB/s) driven by workflow  DDN Lustre  Collaborative sharing of information  Backup & Archiving and resources  Symantec, Quantum

A*STAR PRIMERGY BX920 S2 at A*STAR  Singapore’s lead government agency  450 server blades (3888 cores)  Fostering world-class scientific research  45 Teraflops peak performance  Biomedical Sciences  91% of Linpack efficiency  Physical Sciences & Engineering  Spurs growth in key economic clusters

Fujitsu and A*STAR (IHPC)  R&D partnership to jointly develop  Applications  Technologies for the use of next-gen. supercomputer in  Computational Fluid Dynamics  Material Sciences

PRIMERGY x86 Clusters Celsius workstations PRIMEHPC FX10 Supercomputers