Jaguar

Jake Baskin '10, Jānis Lībeks '10

Jan 27, 2010 What is it?

Currently the fastest supercomputer in the world, at up to 2.33 PFLOPS, located at Oak Ridge National Laboratory (ORNL).

Leader in "petascale scientific supercomputing". Uses Massively parallel simulations.

Modeling: Climate Supernovas Volcanoes Cellulose

http://www.nccs.gov/wp- content/themes/nightfall/img/jaguarXT5/gallery/jaguar-1.jpg Overview

Processor Specifics Network Architecture Programming Models NCCS networking Spider file system Scalability The guts

84 XT4 and 200 XT5 cabinets XT5 18688 compute nodes 256 service and i/o nodes XT4 7832 compute nodes 116 service and i/o nodes (XT5) Compute Nodes

2 2435 Istanbul (6 core) processors per node 64K L1 instruction cache 65K L1 data cache per core 512KB L2 cache per core 6MB L3 cache per processor (shared) 8GB of DDR2-800 RAM directly attached to each processor by integrated memory controller.

http://www.cray.com/Assets/PDF/products/xt/CrayXT5Brochure.pdf How are they organized?

3-D Torus topology XT5 and XT4 segments are connected by an InfiniBand DDR network 889 GB/sec bisectional bandwidth

http://www.cray.com/Assets/PDF/products/xt/CrayXT5Brochure.pdf Programming Models

Jaguar supports these programming models: MPI (Message Passing Interface) OpenMP (Open Multi Processing) SHMEM (SHared MEMory access library) PGAS (Partitioned global address space) NCCS networking

Jaguar usually performs computations on large datasets. These datasets have to be transferred to ORNL.

Jaguar is connected to ESnet (Energy Sciences Network, scientific institutions) and Internet2 (higher education institutions).

ORNL owns its own optical network that allows 10Gb/s to various locations around the US.

Spider File System

Large scale storage cluster. Uses failover pairs, multiple networking paths and resiliency features to ensure redundancy and fault tolerance.

48 DDN S2A9900s servers provide 240 GB/s of bandwidth, over 10 PB of storage on 13,440 1 terabyte SATA drives.

Each compute node has root access to the filesystem. Scalability

Has been expanded during the last year.

Cray XT5 trays are built to avoid bottlenecks.