Update on Cray Activities in the Earth Sciences

Presented to the 13th ECMWF Workshop on the Use of HPC in Meteorology

3-7 November 2008

Per Nyberg [email protected]

Director, Marketing and Business Development Topics

ƒ General Cray Update ƒ Cray Technology Directions • Overall Technology Focus • Specific Updates on: ƒ Cray – Intel Collaboration ƒ Cray CX1 Announcement ƒ ECOphlex Cooling ƒ Perspectives on Petascale Computing

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 2 General Cray Update HPC is Cray’s Sole Mission

ƒ Significant R&D centered around all aspects of high-end HPC:

• From scalable operating systems to cooling.

ƒ Cray XT MPP architecture has gained mindshare as prototype for highly scalable computing:

• From 10’s TF to Petaflop.

ƒ First Cray Petaflop supercomputer to be delivered by end of 2008 (ORNL) and second by early 2009 (NSF).

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 4 November 08 Cray Inc. Proprietary Slide 5 Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 5 ORNL Petascale Jaguar System

ƒ The National Center for Computational Sciences (NCCS) at ORNL is currently installing a new petascale “Jaguar” system. • XT5 with ECOphlex liquid cooling. ƒ Will enable petascale simulations of: • Climate science • High-temperature superconductors • Fusion reaction for the 100-million-degree ITER reactor ƒ Will be the only open science petascale system in the world when it comes online in 2009.

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 6 ORNL Jaguar System

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 7 Climate Usage of DoE Cray Systems

ƒ DoE / NSF Climate End Station (CES) • An interagency collaboration of NSF and DOE in developing the Community Climate System Model (CCSM) for IPCC AR5. • A collaboration with NASA in carbon data assimilation • A collaboration with university partners with expertise in computational climate research. ƒ DoE / NOAA MoU • DoE to provide NOAA/GFDL with millions of CPU hours. • Climate change and near real-time high-impact NWP research . • Prototyping of advanced high-resolution climate models.

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 8 November 08 Cray Inc. Proprietary Slide 9 WRF 'nature' Benchmark on ORNL Cray XT4

ƒ Key Performance Enabling Technologies: • Efficient algorithmic implementation in user application. • Good scalability at small number of gridpoints per core. ƒ Fundamental Cray MPP architectural features for scalability: • MPP 3-D Torus • Efficient communications layers • Light Weight Kernel / Jitter-free OS

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 10 National Science Foundation – Track 2 Award

ƒ Awarded to the University of Tennessee. ƒ Quad-core AMD Opteron processor system to be installed in 2008. ƒ Scheduled upgrade to near petascale in 2009. ƒ Housed at the University of Tennessee – Oak Ridge National Laboratory Joint Institute for Computational Sciences.

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 11 United Kingdom’s Engineering and Physical Sciences Research Council (EPSRC) ƒ Europe’s largest HPC initiative - HECToR (High End Computing Terascale Resources) ƒ Multi-year $85M contract for advanced Cray hybrid system: • Cray XT4 + Cray X2 • Initial peak capability of 60 Tflops ƒ Installed at the University of Edinburgh’s Parallel Computing Centre (EPCC) ƒ ~20% of the system to be used by NERC (Natural Environment Research Council) in areas such as climate research.

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 12 MeteoSwiss and University of Bergen

ƒ In September 2007 CSCS began provision of a Cray XT4 for exclusive operational NWP usage by MeteoSwiss. ƒ By early 2008 MeteoSwiss was the first European country to run 1~2 km high resolution regional forecasts. ƒ 52 Tflops XT4 installed in 2008. ƒ Primary disciplines include marine molecular biology, oceanography and climate research. ƒ Users include: ƒ University of Bergen ƒ Institute of Marine Research ƒ Nansen Environmental and Remote Sensing Center ƒ NOTUR

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 13 Danish Meteorological Institute

ƒ DMI win was announced in November 2007. ƒ Will significantly enhance DMI’s NWP and climate assessment capabilities. ƒ Dual XT5 system configuration for operations and research with full failover. ƒ Current operations has transitioned from previous platform to running on single cabinet XT4. ƒ Will transition next operational suite to XT5s in early 2009.

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 14 Cray Technology Directions Opposing Forces in Today’s HPC Marketplace

29,712

16,316 Supercomputing is increasingly about managing massive parallelism: • Key to overall system performance, not only 10,073 individual application performance. • Demands investment in technologies to effectively harness parallelism (hardware, software, I/O, management, etc…). 3,518 2,827 3,093 2,230 1,644 1,847 1,245 808 1,073 408 722

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 Average Number of Processors Per Supercomputer (Top 20 of Top 500) (Nov) Source: www.top500.org Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 16 “Granite” The Cray Roadmap “Baker”+ Realizing Our Adaptive Supercomputing Vision “Marble”

2 0 “Baker” 1 1 +

Cray XT5 2 0 1 & XT5h 0

2 00 Cray XT4 9

CX1

Scalar Cray XMT 20 Vector 08

Multithreaded Nov-2008 Deskside Slide 17 Cray Technology Directions ƒ Scalable System Performance: • Enabling scalability through technologies to manage and harness parallelism. • In particular interconnect and O/S technologies. ƒ Cray is a pioneer and leader in light weight kernel design.

ƒ Green Computing: packaging, power consumption and ECOphlex cooling: • Simple expansion and in-cabinet upgrade to latest technologies, preserving investment in existing infrastructure. • Lowest possible total electrical footprint.

ƒ Provide investment protection and growth path through R&D into future technologies: HPCS Cascade Program.

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 18 Cray and Intel to Collaborate to Develop Future Supercomputing Technologies ƒ Announced on April 28, 2008. ƒ Comprehensive multi-year agreement to advance high-performance computing on Intel processors while delivering new Intel and Cray technologies in future Cray systems. ƒ Greater choice for Cray customers in processor technologies. ƒ Cray and Intel will explore future supercomputer component designs such as multi-core processing and advanced interconnects. ƒ First HPC products will be available in the Cascade timeframe.

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 19 Cray CX1 Deskside Supercomputer

ƒ Certified Cray Quality ƒ Best in Class Customer Experience ƒ Integrated HPC Operating Environment

ƒ World’s Most Versatile Cluster ƒ Single or dual socket compute node ƒ High-end graphic node ƒ High-capacity storage node ƒ Up to 4TB of internal storage ƒ Built-in Gigabit or InfiniBand Interconnect ƒ Office and lab compatible We Take Supercomputing Personally with standard office power outlet Two Approaches to Cooling – Single Infrastructure High Efficiency Cabinet can be Liquid or Air-Cooled

ƒ Liquid Cooled – HE Cabinet with ECOphlex Technology • No change in ambient room temperature. • Minimizes power consumption by reducing need for air handler/conditioners. • Elimination of Chillers can be the single biggest step toward “Green” • Critical for situations where little or no additional air cooling is available. ƒ Air Cooled – HE Cabinet • Utilizing each cubic foot of cooling air to the extreme • Minimal total volume of cooling air required. • Minimal fan electrical power consumed as compared to muffin fans. ƒ Made possible by • Proprietary Axial Turbine Blower • Progressive heat sink technology blade design • Progressive air cooled chassis design.

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 21 PHase change Liquid EXchange(PHLEX)

Cool air is released into the computer room

Liquid Liquid/Vapor in Mixture out

Hot air stream passes through evaporator, rejects heat to R134a via liquid-vapor phase change (evaporation).

ƒ R134a absorbs energy only in the presence of heated air. ƒ Phase change is 10x more efficient than pure water cooling. ƒ Corollary: Weight of coils, fluid, etc. is 10X less than water cooling

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 22 XT5 System with ECOphlex

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 23 Advantages of ECOphlex vs. Water-Cooled Solutions ƒ No water near computer components • If leaked, will not damage components • No condensate drains or chance of water damage ƒ Lightweight • Small volume of coolant in the compute cabinet. • Floor load is similar to air-cooled cabinets ƒ Cost of Ownership • Coolant can be re-condensed using building water. • In many cases, cooling can be “free” ƒ Serviceability and Reliability • Blades are still air cooled and can be pulled while system in operation. • Large systems can continue to fully function if an HEU is down.

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 24 Perspectives on Petascale Computing Petascale (or even Terascale) Perspectives ƒ There remains a tremendous number of models and application areas that have not yet reached even the terascale level. ƒ Those that can scale have benefited from a focused, iterative multi-year algorithmic optimization effort: ƒ Optimization strategies do not remain stagnant and must take advantage of evolving hardware and software technologies. ƒ Ongoing access to scalable, leadership class systems and support is essential: • Either you have actually run on 5K, 10K, 20K, 50K… processors, or you have not ! Theory vs. Reality

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 26 Evolving Path to Petascale

ƒ Three basic paths are available to users today: • Standards based MPP • Low power • Accelerators ƒ Cray Cascade program is intended to bring these together to provide greater flexibility within a single architecture. ƒ Will be accomplished through: • Continued investment in core MPP and emerging technologies. • Continued investment in customer relationships. ƒ Cray’s wealth of experience in pushing the boundaries of scalability will continue to positively impact entire HPC community.

Nov-2008 13th ECMWF Workshop on the Use of HPC in Meteorology Slide 27 Thank you for your attention Thank you for your attention.

17-Sep-2007 CSCS XT4 Inauguration Slide 28