SUSE® High Performance Computing Kai Dupke Meike Chabowski Senior Product Manager Senior Product Marketing Manager SUSE Linux Enterprise SUSE Linux Enterprise [email protected] [email protected] Distribution: pdf any Date: 2015-10-22 Not a public document. 3 HPC Impacts Our Lives – Space Research 4 5 HPC Impacts Our Lives – Weather & Climate 6 7 HPC Impacts Our Lives – Oil & Gas 8 9 HPC Impacts Our Lifes - Entertainment 10 From Cinema ... … to Games 11 High Productivity Computing Big Data – or HPC?? 12 Overview HPC Overview SUSE® High Performance Computing • Solving computational, data-intensive, or numerically-intensive tasks • Reducing the time and effort required to set-up and maintain HPC clusters • Ensuring that all components of the HPC stack work together 14 HPC Development SUSE® High Performance Computing • Yesterday ‒ Academia and Research • Today ‒ Academia and Research ‒ Financial Services ‒ Oil and Gas ‒ Semiconductor ‒ Life Sciences ‒ Manufacturing • Tomorrow ‒ Departmental and workgroup clusters ‒ High Productivity Computing 15 Market Segmentation SUSE High Performance Computing System HPC class Ready Key drivers GTM Budget Super Computer >500K$ +++ Special build HW HPC-IHV 'top 500' only performance count Self-supported Partner supported Divisional <500K$ +++ Customized HW IHV Partner driven ISV Partner supported SI SUSE supported Departmental <250K$ ++ Commodity HW Channel Business driven SUSE SUSE supported Work Group <100K$ + Customer driven Channel Home brewed Shop 16 Split Market SUSE® HPC Commercial Scientific High Productivity Top 500-class Computing • Lighthouse projects • Highly specialized application • Government sponsored • ROI and reliability • Generic workloads are key • Often self-supported by • Data Center support Academic staff • Commodity hardware • Specialized hardware 17 SUSE Linux Enterprise HPC SUSE – Strong in HPC Market! SUSE® HPC MULTI- and MANY-CORE PROCESSOR SUPPORT Intel, AMD, POWER ….. COOPERATION TECHNOLOGY IBM Bull Kernel 3.x SUSE SGI NEC Lustre enablement Since 1992 HP Ceph storage platform Cray up to 8192 cores Strong Dell Cisco ….. in Top500 ….. ….. ACADEMIC HIGH PRODUCTIVITY AND RESEARCH COMPUTING LRZ / SuperMUC Total BSC / MareNostrum Baker Hughes Tokyo Institute of Technology Texas Instruments Beijing Computing Center ….. NASA ….. 19 Why Linux? SUSE® High Performance Computing • Open Source benefits ‒ Easy to customize, maintain and improve • Innovation ‒ Beowulf Clusters “born” on Linux • Modularity ‒ GUI overhead not required ‒ appliance form factors • Linux Standards ‒ Large base of tools, including remote management ‒ Hardware availability ‒ Large vendor ecosystem surrounding Linux HPC clusters 20 Linux Preferred for HPC SUSE® High Performance Computing • Linux ‒ runs on more than 97% of the world's top 500 supercomputers* ‒ is used by nearly 90% of general clusters ‒ Linux is used in the majority of HPC systems, from smaller departmental implementations to larger, integrated cluster solutions *top500.org July 2015 21 Why SUSE® Linux Enterprise Server For High Performance Computing • Early player in HPC, pushing innovation and new technologies • Highly reliable, interoperable and manageable server operating system • Built to power mission-critical workloads in physical, virtual and cloud environments • The natural successor to UNIX, backed by proven services for UNIX migration • Special features to improve performance • Backed by established ecosystem – support and certificates • The only Linux recommended by Microsoft 22 SUSE Additional Features SUSE® High Performance Computing • Up-to-date Linux Kernel for optimal performance • CPU Management and System Activity ‒ CPUset System, CPUset command line tool ‒ Sysstat package ‒ IRQbalance • OpenFabrics Enterprise Distribution (OFED) ‒ Remote Direct Memory Access (RDMA) switched fabric technologies, high-speed data transport technologies for server and storage connectivity • SystemTap, LTTng 2.0 • Lustre enabled Kernel 23 SUSE Advanced I/O Processing SUSE® High Performance Computing • Asynchronous I/O (AIO) ‒ Input/output processing that permits other processing to continue before the transmission has finished • Modular I/O Scheduler ‒ Algorithm most suitable for workload can be chosen dynamically • Multi-core/hyper-threading processor support ‒ Execute threads in parallel within each individual processor ‒ Supports up to 4096 cores per system • Intel I/O Acceleration ‒ Offloads the CPU towards the network card, thus allowing the system to continue processing data while I/O is taking place 24 Update Simplify projects! SUSE® HPC • Simplified model ‒ Number of socket pairs matter ‒ Socket pairs are accumulated per system ‒ Head nodes and compute nodes are threaten equal 26 Example – regular HPC setup SUSE® HPC • 2 (8 sockets) head nodes for redundancy / scalability 100 (4 sockets) compute nodes • 416 sockets total (order: 208x 1-2 sockets) Head Node Compute Nodes total Client 2x 8 sockets + 100x 4 sockets = 416 sockets 27 Keep it running! SUSE® HPC • SUSE Vendor Support ‒ Maintenance ‒ Standard & Priority support for the whole system 29 Recent Developments SUSE® High Performance Computing • Storage • SUSE Linux Enterprise ‒ Release of SUSE Storage ‒ 11 SP4 with latest HW with Infiniband support enablement (Intel, Power8) • ARM64 • Network ‒ Partnering with Cavium ‒ Higher network throughput ‒ SUSE Linux Enterprise for ‒ Added tunables in the IP ARM64 stack (for lower latency) • Cloud ‒ MS Azure with SLES 12: RDMA & Infiniband 30 Partner, Customers Customers and Partners SUSE® High Performance Computing Customers Partners 32 PANGEA – Total Exploration SUSE High Performance Computing • Oil & Gas exploration ‒ Process seismic data ‒ Simulation of deposit fluids • Superior Performance “... from our point of view, SGI plus SUSE Linux ‒ 2.3 petaflops Enterprise Server was a complete, integrated ‒ 10x performance increase solution.” ‒ Equivalent to 27,000 PCs “... SUSE Linux Enterprise Server gives us the ability to • Future is coming keep scaling on ever larger machines.” ‒ 6.7 petaflops — Diego Klahr HPC Engineer Total ‒ Equivalent to 80,000 PCs 33 Intel Cluster Ready Program SUSE® High Performance Computing • Designed to simplify purchasing, deployment and management of HPC clusters • SUSE Linux Enterprise Server is Intel Cluster Ready and powers many certified Intel Cluster Ready systems • intel® Cluster Ready “recipes” are available with SUSE Linux Enterprise Server ‒ Reference designs to help hardware vendors, platform integrators, and system integrators design and build certified Intel Cluster Ready systems 36 Outlook Challenge SUSE® High Performance Computing HPC market fast developing Stack components provided by various vendors Some stack components run in parallel Mix of small and big vendors Segmented into commercial and scientific 38 Outlook SUSE® High Performance Computing • SUSE Linux Enterprise ‒ 12 SP1 beta program running • SUSE HPC ‒ Evaluating optimized SUSE Linux Enterprise for HPC ‒ Your input is needed! 39 Forward looking statement, might change without notice. Learn more www.suse.com/products/server/hpc.html Thank you. Kai Dupke Meike Chabowski Senior Product Manager Senior Product Marketing Manager SUSE Linux Enterprise SUSE Linux Enterprise [email protected] [email protected] 40 Backup HPC Stack HPC Stack SUSE® High Performance Computing Application Queuing / Management Software & Tools PBS Pro Moab IBM LSF Bright CM Message Passing Interface Storage Network MPI EXT3 XFS BTRFS 10G OFED Parastation Intel HP SGI OCFS2 NFS pNFS TCP offload MPICH openMPI IBRIX GPFS pNFS Lustre cephFS SUSE Linux Enterprise Server Hardware = SUSE supported = SUSE Partner = SUSE future 43 SuperMUC SuperMUC – Facts SUSE® High Performance Computing • 60x faster, one of the fastest HPC systems in Europe • 20x better performance per Watt, provide green HPC • > 155,000 Intel Xeon Processor, migration from Itanium2 to x86 45 LRZ - Leibniz Rechenzentrum Europe’s supercomputer run SUSE Linux Enterprise Server Business challenge: LRZ is part of the Gauss Centre for Supercomputing (GCS), which “We have relied on SUSE operates the most powerful HPC infrastructure in Europe, and needs to Linux Enterprise Server for 15 provide researchers across Europe with a reliable and powerful HPC platform, which enables users to make faster progress in their complex years, and have always been research projects. To reduce the environmental impact of HPC, the institution very satisfied. aimed at improving the energy efficiency leverage established automation solutions to maximise the efficiency and manageability of the new The SUSE team is close at supercomputing platform. hand, should we require support or guidance. Solution: Working with SUSE and IBM, LRZ implemented SuperMUC with approx. 9,400 general purpose computing nodes, a peak performance of three We have received highly Petaflop/s, comprised of 155,000 Intel Xeon processor cores and more competent support over the than 300 TB main memory. LRZ chose to run SuperMUC on SUSE Linux years, and look forward to Enterprise Server, leveraging SUSE’s proven HPC expertise and leading collaborating with them. automation tools such as AutoYaST, which allows systems to be installed without manual intervention. — Dr. Herbert Huber Division Head of Supercomputing Benefits: Leibniz Rechenzentrum • Completed easy and smooth migration from previous Itanium 2 infrastructure to new x86 processor architecture • Considerably simplified configuration and automation of the new system, using the automation