Cray XE™ and XK™ Application Programming and Optimization Student Guide
Total Page:16
File Type:pdf, Size:1020Kb
Cray XE™ and XK™ Application Programming and Optimization Student Guide TR-XEP_BW This document is intended for instructional purposes. Do not use it in place of Cray reference documents. Cray Private © Cray Inc. All Rights Reserved. Unpublished Private Information. This unpublished work is protected by trade secret, copyright, and other laws. Except as permitted by contract or express written permission of Cray Inc., no part of this work or its content may be used, reproduced, or disclosed in any form. U.S. GOVERNMENT RESTRICTED RIGHTS NOTICE: The Computer Software is delivered as "Commercial Computer Software" as defined in DFARS 48 CFR 252.227-7014. All Computer Software and Computer Software Documentation acquired by or for the U.S. Government is provided with Restricted Rights. Use, duplication or disclosure by the U.S. Government is subject to the restrictions described in FAR 48 CFR 52.227-14 or DFARS 48 CFR 252.227-7014, as applicable. Technical Data acquired by or for the U.S. Government, if any, is provided with Limited Rights. Use, duplication or disclosure by the U.S. Government is subject to the restrictions described in FAR 48 CFR 52.227-14 or DFARS 48 CFR 252.227-7013, as applicable. Autotasking, Cray, Cray Channels, Cray Y-MP, and UNICOS are federally registered trademarks and Active Manager, CCI, CCMT, CF77, CF90, CFT, CFT2, CFT77, ConCurrent Maintenance Tools, COS, Cray Ada, Cray Animation Theater, Cray APP, Cray Apprentice2, Cray Apprentice2 Desktop, Cray C90, C90D, Cray C++ Compiling System, Cray CF90, Cray CX1, Cray EL, Cray Fortran Compiler, Cray J90, Cray J90se, Cray J916, Cray J932, Cray Linux Environment, Cray MTA, Cray MTA-2, Cray MTX, Cray NQS, Cray Research, Cray SeaStar, Cray SeaStar2, Cray SeaStar2+, Cray SHMEM, Cray S-MP, Cray SSD-T90, Cray SuperCluster, Cray SV1, Cray SV1ex, Cray SX-5, Cray SX-6, Cray T3D, Cray T3D MC, Cray T3D MCA, Cray T3D SC, Cray T3E, Cray T90, Cray T916, Cray T932, Cray Threadstorm, Cray UNICOS, Cray X1, Cray X1E, Cray X2, Cray XD1, Cray X-MP, Cray XMS, Cray XMT, Cray XR1, Cray XT, Cray XT3, Cray XT4, Cray XT5, Cray XT5h, Cray XT5m, Cray XT6, Cray XE6, Cray XE, Cray Y-MP EL, Cray-1, Cray-2, Cray-3, CrayDoc, CrayLink, Cray-MP, CrayPacs, CrayPat, CrayPort, Cray/REELlibrarian, CraySoft, CrayTutor, CRInform, CRI/TurboKiva, CSIM, CVT, Delivering the power…, Dgauss, Docview, ECOphlex, EMDS, GigaRing, HEXAR, HSX, IOS, ISP/Superlink, LibSci, MPP Apprentice, ND Series Network Disk Array, Network Queuing Environment, Network Queuing Tools, NodeKARE, OLNET, RapidArray, RQS, SEGLDR, SMARTE, SSD, SUPERLINK, System Maintenance and Remote Testing Environment, Trusted UNICOS, TurboKiva, UNICOS MAX, UNICOS/lc, UNICOS/mk and UNICOS/mp are trademarks of Cray Inc. The UNICOS, UNICOS/mk, and UNICOS/mp operating systems are derived from UNIX System V. These operating systems are also based in part on the Fourth Berkeley Software Distribution (BSD) under license from The Regents of the University of California. All other trademarks are the property of their respective owners. Direct comments about this publication to: Mail: Cray Inc. Customer Documentation and Training P.O. Box 6000 Chippewa Falls, WI 54729-0080 USA E-mail: [email protected] Fax: +1 715 726 4991 Cray XE and XK Application Programming and Optimization Student Guide Table of Contents Cray XE and XT Systems Application Programming and Optimization ........................... 9 Course Description........................................................................................................ 10 Topic and Daily Schedule ............................................................................................ 11 Reference Documents ................................................................................................... 12 Cray System Overview ..................................................................................................... 13 Overview Topics ........................................................................................................... 14 Cray System .................................................................................................................. 15 System Overview .......................................................................................................... 16 System Overview (External Services) .......................................................................... 17 Cabinet .......................................................................................................................... 18 Liquid Cooled Cabinets ................................................................................................ 19 Blades ............................................................................................................................ 20 Cray XT and Cray XE Systems .................................................................................... 21 XT5 Compute Blade ..................................................................................................... 22 XT5 Node Block Diagram ............................................................................................ 23 X6 Compute Blade with Gemini ASICs ....................................................................... 24 XE6 Node Block Diagram ............................................................................................ 25 XK6 Compute Blade ..................................................................................................... 26 NVIDIA Tesla™ X2090 GPU ...................................................................................... 27 XIO Service Blades....................................................................................................... 28 XIO Blade ..................................................................................................................... 29 SeaStar ASIC ................................................................................................................ 30 SeaStar Diagram ........................................................................................................... 31 Node Connection Bandwidths ...................................................................................... 32 Portals API Design ........................................................................................................ 33 Portals API Design ........................................................................................................ 34 Gemini........................................................................................................................... 35 Performance and Resiliency Features ........................................................................... 36 Terminology .................................................................................................................. 37 Gemini Block Diagram ................................................................................................. 38 Gemini vs SeaStar – Topology ..................................................................................... 39 Node Types ................................................................................................................... 40 Service Nodes ............................................................................................................... 41 Service Nodes ............................................................................................................... 42 Service Node Software Components ............................................................................ 43 System Networks .......................................................................................................... 44 Network Topology Classes ........................................................................................... 45 A Single-chassis Configuration .................................................................................... 46 Class 0 Cable Drawing, 3 Cabinets............................................................................... 47 Class 1 Cable Drawing, 4 Cabinets............................................................................... 48 Class 2 Cable Drawing, 16 Cabinets............................................................................. 49 Class 3 Topology .......................................................................................................... 50 HSN Cabling Photo....................................................................................................... 51 Cray XT5m Topology ................................................................................................... 52 TR-XEP_BW Cray Private 1 Cray XE and XK Application Programming and Optimization Student Guide Identifying Components................................................................................................ 53 Node ID (NID) .............................................................................................................. 54 Cabinet 0 NID Numbering Example ............................................................................. 55 DVS, DSL (Dynamic Shared Libraries), and Cluster Compatibility Mode (CCM) ......... 57 DVS............................................................................................................................... 58 DVS............................................................................................................................... 59 Dynamic Shared Objects and Libraries (DSL) ............................................................