Confidential

Industry-Standard Green HPC Systems

HPC Advisory Council Brazil Conference 2014

May 26, 2014 University of São Paulo

Attila A. Nagy Senior IT Consultant

© Supermicro 2014 Confidential HPC systems: What the industry is doing

Architecture Processor

0.4 15.4 8.0 0.6 Cluster 8.6 Opteron MPP Power 84.6 82.4 Sparc Other

system share (%) system share (%)

Source: The Top500 list of November 2013. http://www.top500.org Confidential HPC systems: What the industry is doing

Interconnect Operating System

2.2 1.4 4.0 2.2 Infiniband Linux 10.0 GbE 15.4 41.4 Unix 10GbE Other 96.4 Custom 27.0 Cray system share (%) Other system share (%)

Source: The Top500 list of November 2013. http://www.top500.org Confidential Industry-standard HPC clusters

 Standard x86 servers throughout . Compute nodes . Head/Management/Control nodes . Storage nodes  Infiniband and/or Ethernet networks . Main interconnect . Cluster management and administration . Out-of-band management  Linux OS environment . Comprehensive software stack for HPC . Large availability of HPC software tools . Large collaboration community Confidential Typical HPC Cluster

IB Fabric Head & Management Nodes Campus Ethernet Fabric x86 servers/Linux Network

Infiniband The Beowulf cluster concept Ethernet OOB Mgmt Network is as solidNetwork as ever!! Network

Storage Nodes Compute Nodes x86 servers x86 servers Linux Linux Parallel FS Confidential Accelerator/Coprocessor usage

 All top ten systems in the latest Green500 list are coprocessor Accelerator/CP based* . Two petaflop systems

N/A  Up to 5x improvements in: 89.4 Nvidia . Power consumption . Physical space Xeon Phi 7.4 . Cost Other 0.8 2.4

 GPUs/MICs: 80% of HPC users system share (%) at least testing them

Source: The Top500 list of November 2013. http://www.top500.org

(*) The Green500 list of November 2013. http://www.green500.org Confidential Green HPC?

 Data centers consumed approximately 1.3% of worldwide electricity production in 2010 (2% in the US) ¹

 Data center power usage growth 2012-2013 (%): ² 15.1

10.9  Power is not the only issue: 7.2 . Cooling 6.7 . Real estate . Management Brazil LatAm US World  Cost (OpEx)

(1) Koomey, Jonathan. 2011. Growth in Data center electricity use 2005 to 2010. Oakland, CA: Analytics Press. August 1. http://www.analyticspress.com/datacenters.html

(2) DCD Industry Census 2013: Data Center Power. January 31, 2014 by DCD Intelligence. http://www.datacenterdynamics.com/focus/archive/2014/01/dcd-industry-census-2013-data-center-power Confidential The roadmap to Green HPC

 Processing efficiency Power consumption . Heterogeneous computing breakdown

4% 6% GPU 100%  Server design 6% 95% CPU . Power 90% 9% 85% Memory

. Thermal 80% Efficiency 50%75% Fans . Mechanical 70% PWS-920P-1R Platinum PWS-721P-1R Gold 65% Power Supply Previous Generation 25% 60% 0% 20% 40% 60% 80% 100% Loading Other

Node configuration: 2x Xeon E5v2; 2x Nvidia Kepler; 128GB memory; no HDD Confidential The roadmap to Green HPC

 Data center design . PUE (Power Usage Effectiveness) . Layout . AC/cooling

 Management . Server, rack/system & data center level . Core on/off & speed control . Power monitoring/capping/policy setting . Agent based & OOB . Integrated data center mgmt & control Cost Efficiency

~25kW Air cool Liquid cool

KW / rack Confidential Supermicro HPC

 Innovation – HPC Optimized H/W Design and Manufacturing

 Offer broadest line of x86 HPC server building blocks

 Design to fit any HPC application – all about choices

 Focus on high efficiency () to enable green HPC and scalability

 Channel Enablement – First to Market with Building Block Solutions

 Provide first to market competitiveness to our partners: HPC solution providers

 Enable channel partners with fully validated HPC building blocks

 End users get a solution that is optimized for their applications

 Commoditization

 Driving the technology curve from proprietary to open architecture and commodity

 Commitment to the HPC community to make high-performance solutions affordable in order to accelerate science & engineering research and development

Confidential

Thank you!

Attila A. Nagy [email protected] www.supermicro.com