Jaguar: the World's Most Powerful Computer
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Petaflops for the People
PETAFLOPS SPOTLIGHT: NERSC housands of researchers have used facilities of the Advanced T Scientific Computing Research (ASCR) program and its EXTREME-WEATHER Department of Energy (DOE) computing predecessors over the past four decades. Their studies of hurricanes, earthquakes, NUMBER-CRUNCHING green-energy technologies and many other basic and applied Certain problems lend themselves to solution by science problems have, in turn, benefited millions of people. computers. Take hurricanes, for instance: They’re They owe it mainly to the capacity provided by the National too big, too dangerous and perhaps too expensive Energy Research Scientific Computing Center (NERSC), the Oak to understand fully without a supercomputer. Ridge Leadership Computing Facility (OLCF) and the Argonne Leadership Computing Facility (ALCF). Using decades of global climate data in a grid comprised of 25-kilometer squares, researchers in These ASCR installations have helped train the advanced Berkeley Lab’s Computational Research Division scientific workforce of the future. Postdoctoral scientists, captured the formation of hurricanes and typhoons graduate students and early-career researchers have worked and the extreme waves that they generate. Those there, learning to configure the world’s most sophisticated same models, when run at resolutions of about supercomputers for their own various and wide-ranging projects. 100 kilometers, missed the tropical cyclones and Cutting-edge supercomputing, once the purview of a small resulting waves, up to 30 meters high. group of experts, has trickled down to the benefit of thousands of investigators in the broader scientific community. Their findings, published inGeophysical Research Letters, demonstrated the importance of running Today, NERSC, at Lawrence Berkeley National Laboratory; climate models at higher resolution. -
Avionics Thermal Management of Airborne Electronic Equipment, 50 Years Later
FALL 2017 electronics-cooling.com THERMAL LIVE 2017 TECHNICAL PROGRAM Avionics Thermal Management of Advances in Vapor Compression Airborne Electronic Electronics Cooling Equipment, 50 Years Later Thermal Management Considerations in High Power Coaxial Attenuators and Terminations Thermal Management of Onboard Charger in E-Vehicles Reliability of Nano-sintered Silver Die Attach Materials ESTIMATING INTERNAL AIR ThermalRESEARCH Energy Harvesting ROUNDUP: with COOLING TEMPERATURE OCTOBERNext Generation 2017 CoolingEDITION for REDUCTION IN A CLOSED BOX Automotive Electronics UTILIZING THERMOELECTRICALLY ENHANCED HEAT REJECTION Application of Metallic TIMs for Harsh Environments and Non-flat Surfaces ONLINE EVENT October 24 - 25, 2017 The Largest Single Thermal Management Event of The Year - Anywhere. Thermal Live™ is a new concept in education and networking in thermal management - a FREE 2-day online event for electronics and mechanical engineers to learn the latest in thermal management techniques and topics. Produced by Electronics Cooling® magazine, and launched in October 2015 for the first time, Thermal Live™ features webinars, roundtables, whitepapers, and videos... and there is no cost to attend. For more information about Technical Programs, Thermal Management Resources, Sponsors & Presenters please visit: thermal.live Presented by CONTENTS www.electronics-cooling.com 2 EDITORIAL PUBLISHED BY In the End, Entropy Always Wins… But Not Yet! ITEM Media 1000 Germantown Pike, F-2 Jean-Jacques (JJ) DeLisle Plymouth Meeting, PA 19462 USA -
Technical Manual for a Coupled Sea-Ice/Ocean Circulation Model (Version 3)
OCS Study MMS 2009-062 Technical Manual for a Coupled Sea-Ice/Ocean Circulation Model (Version 3) Katherine S. Hedström Arctic Region Supercomputing Center University of Alaska Fairbanks U.S. Department of the Interior Minerals Management Service Anchorage, Alaska Contract No. M07PC13368 OCS Study MMS 2009-062 Technical Manual for a Coupled Sea-Ice/Ocean Circulation Model (Version 3) Katherine S. Hedström Arctic Region Supercomputing Center University of Alaska Fairbanks Nov 2009 This study was funded by the Alaska Outer Continental Shelf Region of the Minerals Manage- ment Service, U.S. Department of the Interior, Anchorage, Alaska, through Contract M07PC13368 with Rutgers University, Institute of Marine and Coastal Sciences. The opinions, findings, conclusions, or recommendations expressed in this report or product are those of the authors and do not necessarily reflect the views of the U.S. Department of the Interior, nor does mention of trade names or commercial products constitute endorsement or recommenda- tion for use by the Federal Government. This document was prepared with LATEX xfig, and inkscape. Acknowledgments The ROMS model is descended from the SPEM and SCRUM models, but has been entirely rewritten by Sasha Shchepetkin, Hernan Arango and John Warner, with many, many other con- tributors. I am indebted to every one of them for their hard work. Bill Hibler first came up with the viscous-plastic rheology we are using. Paul Budgell has rewritten the dynamic sea-ice model, improving the solution procedure and making the water- stress term implicit in time, then changing it again to use the elastic-viscous-plastic rheology of Hunke and Dukowicz. -
Introducing the Cray XMT
Introducing the Cray XMT Petr Konecny Cray Inc. 411 First Avenue South Seattle, Washington 98104 [email protected] May 5th 2007 Abstract Cray recently introduced the Cray XMT system, which builds on the parallel architecture of the Cray XT3 system and the multithreaded architecture of the Cray MTA systems with a Cray-designed multithreaded processor. This paper highlights not only the details of the architecture, but also the application areas it is best suited for. 1 Introduction past two decades the processor speeds have increased several orders of magnitude while memory speeds Cray XMT is the third generation multithreaded ar- increased only slightly. Despite this disparity the chitecture built by Cray Inc. The two previous gen- performance of most applications continued to grow. erations, the MTA-1 and MTA-2 systems [2], were This has been achieved in large part by exploiting lo- fully custom systems that were expensive to man- cality in memory access patterns and by introducing ufacture and support. Developed under code name hierarchical memory systems. The benefit of these Eldorado [4], the Cray XMT uses many parts built multilevel caches is twofold: they lower average la- for other commercial system, thereby, significantly tency of memory operations and they amortize com- lowering system costs while maintaining the sim- munication overheads by transferring larger blocks ple programming model of the two previous genera- of data. tions. Reusing commodity components significantly The presence of multiple levels of caches forces improves price/performance ratio over the two pre- programmers to write code that exploits them, if vious generations. they want to achieve good performance of their ap- Cray XMT leverages the development of the Red plication. -
Lessons Learned in Deploying the World's Largest Scale Lustre File
Lessons Learned in Deploying the World’s Largest Scale Lustre File System Galen M. Shipman, David A. Dillow, Sarp Oral, Feiyi Wang, Douglas Fuller, Jason Hill, Zhe Zhang Oak Ridge Leadership Computing Facility, Oak Ridge National Laboratory Oak Ridge, TN 37831, USA fgshipman,dillowda,oralhs,fwang2,fullerdj,hilljj,[email protected] Abstract 1 Introduction The Spider system at the Oak Ridge National Labo- The Oak Ridge Leadership Computing Facility ratory’s Leadership Computing Facility (OLCF) is the (OLCF) at Oak Ridge National Laboratory (ORNL) world’s largest scale Lustre parallel file system. Envi- hosts the world’s most powerful supercomputer, sioned as a shared parallel file system capable of de- Jaguar [2, 14, 7], a 2.332 Petaflop/s Cray XT5 [5]. livering both the bandwidth and capacity requirements OLCF also hosts an array of other computational re- of the OLCF’s diverse computational environment, the sources such as a 263 Teraflop/s Cray XT4 [1], visual- project had a number of ambitious goals. To support the ization, and application development platforms. Each of workloads of the OLCF’s diverse computational plat- these systems requires a reliable, high-performance and forms, the aggregate performance and storage capacity scalable file system for data storage. of Spider exceed that of our previously deployed systems Parallel file systems on leadership-class systems have by a factor of 6x - 240 GB/sec, and 17x - 10 Petabytes, traditionally been tightly coupled to single simulation respectively. Furthermore, Spider supports over 26,000 platforms. This approach had resulted in the deploy- clients concurrently accessing the file system, which ex- ment of a dedicated file system for each computational ceeds our previously deployed systems by nearly 4x. -
Titan: a New Leadership Computer for Science
Titan: A New Leadership Computer for Science Presented to: DOE Advanced Scientific Computing Advisory Committee November 1, 2011 Arthur S. Bland OLCF Project Director Office of Science Statement of Mission Need • Increase the computational resources of the Leadership Computing Facilities by 20-40 petaflops • INCITE program is oversubscribed • Programmatic requirements for leadership computing continue to grow • Needed to avoid an unacceptable gap between the needs of the science programs and the available resources • Approved: Raymond Orbach January 9, 2009 • The OLCF-3 project comes out of this requirement 2 ASCAC – Nov. 1, 2011 Arthur Bland INCITE is 2.5 to 3.5 times oversubscribed 2007 2008 2009 2010 2011 2012 3 ASCAC – Nov. 1, 2011 Arthur Bland What is OLCF-3 • The next phase of the Leadership Computing Facility program at ORNL • An upgrade of Jaguar from 2.3 Petaflops (peak) today to between 10 and 20 PF by the end of 2012 with operations in 2013 • Built with Cray’s newest XK6 compute blades • When completed, the new system will be called Titan 4 ASCAC – Nov. 1, 2011 Arthur Bland Cray XK6 Compute Node XK6 Compute Node Characteristics AMD Opteron 6200 “Interlagos” 16 core processor @ 2.2GHz Tesla M2090 “Fermi” @ 665 GF with 6GB GDDR5 memory Host Memory 32GB 1600 MHz DDR3 Gemini High Speed Interconnect Upgradeable to NVIDIA’s next generation “Kepler” processor in 2012 Four compute nodes per XK6 blade. 24 blades per rack 5 ASCAC – Nov. 1, 2011 Arthur Bland ORNL’s “Titan” System • Upgrade of existing Jaguar Cray XT5 • Cray Linux Environment -
Musings RIK FARROWOPINION
Musings RIK FARROWOPINION Rik is the editor of ;login:. While preparing this issue of ;login:, I found myself falling down a rabbit hole, like [email protected] Alice in Wonderland . And when I hit bottom, all I could do was look around and puzzle about what I discovered there . My adventures started with a casual com- ment, made by an ex-Cray Research employee, about the design of current super- computers . He told me that today’s supercomputers cannot perform some of the tasks that they are designed for, and used weather forecasting as his example . I was stunned . Could this be true? Or was I just being dragged down some fictional rabbit hole? I decided to learn more about supercomputer history . Supercomputers It is humbling to learn about the early history of computer design . Things we take for granted, such as pipelining instructions and vector processing, were impor- tant inventions in the 1970s . The first supercomputers were built from discrete components—that is, transistors soldered to circuit boards—and had clock speeds in the tens of nanoseconds . To put that in real terms, the Control Data Corpora- tion’s (CDC) 7600 had a clock cycle of 27 .5 ns, or in today’s terms, 36 4. MHz . This was CDC’s second supercomputer (the 6600 was first), but included instruction pipelining, an invention of Seymour Cray . The CDC 7600 peaked at 36 MFLOPS, but generally got 10 MFLOPS with carefully tuned code . The other cool thing about the CDC 7600 was that it broke down at least once a day . -
Cray XT and Cray XE Y Y System Overview
Crayyy XT and Cray XE System Overview Customer Documentation and Training Overview Topics • System Overview – Cabinets, Chassis, and Blades – Compute and Service Nodes – Components of a Node Opteron Processor SeaStar ASIC • Portals API Design Gemini ASIC • System Networks • Interconnection Topologies 10/18/2010 Cray Private 2 Cray XT System 10/18/2010 Cray Private 3 System Overview Y Z GigE X 10 GigE GigE SMW Fibre Channels RAID Subsystem Compute node Login node Network node Boot /Syslog/Database nodes 10/18/2010 Cray Private I/O and Metadata nodes 4 Cabinet – The cabinet contains three chassis, a blower for cooling, a power distribution unit (PDU), a control system (CRMS), and the compute and service blades (modules) – All components of the system are air cooled A blower in the bottom of the cabinet cools the blades within the cabinet • Other rack-mounted devices within the cabinet have their own internal fans for cooling – The PDU is located behind the blower in the back of the cabinet 10/18/2010 Cray Private 5 Liquid Cooled Cabinets Heat exchanger Heat exchanger (XT5-HE LC only) (LC cabinets only) 48Vdc flexible Cage 2 buses Cage 2 Cage 1 Cage 1 Cage VRMs Cage 0 Cage 0 backplane assembly Cage ID controller Interconnect 01234567 Heat exchanger network cable Cage inlet (LC cabinets only) connection air temp sensor Airflow Heat exchanger (slot 3 rail) conditioner 48Vdc shelf 3 (XT5-HE LC only) 48Vdc shelf 2 L1 controller 48Vdc shelf 1 Blower speed controller (VFD) Blooewer PDU line filter XDP temperature XDP interface & humidity sensor -
An Expansion on Applied Computer Cooling
An Expansion on Applied Computer Cooling By Spencer Ellsworth LAES 400 November 29, 2012 Abstract In the world of professional technology, many businesses utilize servers and Ap- ple products in their day-to-day work, relying upon these machines for the liveli- hood of their operations. However, the technology being used is limited by a single, ever-important factor: heat. The key to improved performance, increased computer longevity, and lower upkeep costs lies in managing machine temperatures. Previously, this topic was investigated for the world of PCs. In this paper, modifications, upgrades, and available components that improve cooling are discussed in regard to servers and Apple products. Through the use of these methodologies, the aformentioned improve- ments may be achieved. Contents 1 Introduction 1 2 Background 1 3 Servers 2 3.1 Freestanding . 3 3.2 Rack Mount . 3 4 Apple Products 5 4.1 Difficulties . 5 4.2 Mac Desktops . 5 4.2.1 iMac . 6 4.2.2 Mac Pro . 6 4.3 Mac Mini . 7 4.4 Apple TV . 7 4.5 MacBook . 8 5 Business Economics 8 5.1 Servers . 8 5.2 Apple . 9 6 Related Work 9 7 Conclusion 10 1 Introduction freestanding and rack mount servers, as their differences are significant. The topics of \Now this is not the end. It is not even this paper then make a transition to Ap- the beginning of the end. But it is, per- ple products, touching upon the difficulties haps, the end of the beginning." -Sir Win- in improving cooling for Apple machines, as ston Churchill, 1942 well as individual investigations of the iMac, Six months ago, the author completed a Mac Pro, Mac Mini, Apple TV, and Mac- research, analysis, and experimentation pa- Book. -
Motivair Cooling Solutions Report
Cooling System Considerations for High Density Colocation and Enterprise Data Centers By Rich Whitmore Executive summary The recent advances in computer technology and the growing adoption of high performance CPUs and GPUs are allowing users to achieve new heights in computer analytics including the use of Big Data, Artificial Intelligence, High-Frequency Trading, Oil & Gas research and web scale business models. This rapid rise in use has overtaken most Colocation and Enterprise facilities’ ability to cool the se often-densified server racks on a large scale. While many facilities publish the ability to provide cooling for higher than standard server racks on a watts per square foot basis, many if not all are unable to manage newer computer systems at that density on a large scale. Co location and Enterprise data centers must consider how these new computers will interact within the data center environment, educate themselves on the various solutions available to cool these dense servers and build the infrastructure that can support the latest computer racks being used both today and in the future. Cooling System Considerations for High Density Colocation and Enterprise Data Centers Introduction As the uses for high density computers continues to expand, the requirements for operating these advanced systems has also grown. The drive for higher efficiency data centers is a subject closely tied to a building’s Power Usage Effectiveness (PUE) which is defined as: (Total Facility Energy) / (IT Equipment Energy). High Performance Computing (HPC) clusters and high-density compute racks are consuming power densities of up to 100 kW per rack, sometimes higher, with an estimated average density of 35 kW per rack. -
“Cooling System for Electronics in Computer System- an Overview”
International Journal of Recent Engineering Research and Development (IJRERD) Volume No. 02 – Issue No. 02, ISSN: 2455-8761 www.ijrerd.com, PP. 01-04 “Cooling system for electronics in computer system- an overview” KunalPardeshi1, P.G Patil2, R.Y.Patil3 1 Master Scholar MechanicalEnineering Department, S.G.D.C.O.E. Jalgaon 2Assistant Professor, Mechanical Enineering Department, S.G.D.C.O.E. Jalgaon 3Head, Mechanical Department, S.G.D.C.O.E. Jalgaon Abstract: During the power management, the inevitable power losses induce heat generation in the power electronics. In this, effective design for the cooling system is essential in terms of safety, reliability, and durability. A liquid cooling system for the power electronics is applied to chill the electrical components below the thermal specifications. Nonetheless, the layout of cooling components is usually designed after the completion of the chassis and power electronics in the automotive applications, thus, only a little freedom is allowed to change the layout. Thus, it is significant and urgent to investigate the cooling performance before finalizing the layout design. In this work, one dimensional and computerized fluid dynamics code is employed to simulate the performance of the cooling system at the early stage of conceptual design. Three different layouts of cooling systems are chosen to compare the ensuing systematic cooling performances. The liquid flow rates, pressure drops, and maximum temperatures are computed by the numerical simulations of the cooling system which comprises the cold plates, liquid pump and heating electronic device. Index Terms: High performance CPUs, heat transfer enhancement, liquid impingement, heat transfer 1. -
Jaguar Supercomputer
Jaguar Supercomputer Jake Baskin '10, Jānis Lībeks '10 Jan 27, 2010 What is it? Currently the fastest supercomputer in the world, at up to 2.33 PFLOPS, located at Oak Ridge National Laboratory (ORNL). Leader in "petascale scientific supercomputing". Uses Massively parallel simulations. Modeling: Climate Supernovas Volcanoes Cellulose http://www.nccs.gov/wp- content/themes/nightfall/img/jaguarXT5/gallery/jaguar-1.jpg Overview Processor Specifics Network Architecture Programming Models NCCS networking Spider file system Scalability The guts 84 XT4 and 200 XT5 cabinets XT5 18688 compute nodes 256 service and i/o nodes XT4 7832 compute nodes 116 service and i/o nodes (XT5) Compute Nodes 2 Opteron 2435 Istanbul (6 core) processors per node 64K L1 instruction cache 65K L1 data cache per core 512KB L2 cache per core 6MB L3 cache per processor (shared) 8GB of DDR2-800 RAM directly attached to each processor by integrated memory controller. http://www.cray.com/Assets/PDF/products/xt/CrayXT5Brochure.pdf How are they organized? 3-D Torus topology XT5 and XT4 segments are connected by an InfiniBand DDR network 889 GB/sec bisectional bandwidth http://www.cray.com/Assets/PDF/products/xt/CrayXT5Brochure.pdf Programming Models Jaguar supports these programming models: MPI (Message Passing Interface) OpenMP (Open Multi Processing) SHMEM (SHared MEMory access library) PGAS (Partitioned global address space) NCCS networking Jaguar usually performs computations on large datasets. These datasets have to be transferred to ORNL. Jaguar is connected to ESnet (Energy Sciences Network, scientific institutions) and Internet2 (higher education institutions). ORNL owns its own optical network that allows 10Gb/s to various locations around the US.