HPC System Architecture, Programming Models, Compilers

Total Page:16

File Type:pdf, Size:1020Kb

HPC System Architecture, Programming Models, Compilers HPC Programming Models, Compilers, Performance Analysis IBM Systems – Infrastructure Solutions Ludovic Enault, IBM Geoffrey Pascal, IBM 2 Agenda . System architecture trend overview . Programming models & Languages . Compiler . Performance Analysis Tools © 2015 IBM Corporation System architecture trend overview 3 APGAS project review 5IBM2/20/2017 Confidential What’s happening? today Homogeneous Single core Many-core Hybrid Multi-core Heterogeneous GPU/FPGA standard memory model: unique Non-standard memory model: 2 architectures : memory address space separate memory address spaces, Programming Model Complexity . Industry shift to multi-cores/many-cores, accelerators – Intel Xeon+PHI+FPGA, IBM POWER+GPU+FPGA, ARM+GPU+FPGA . Increasing – # Cores – Heterogeneity with Unified Memory – Memory complexity © 2015 IBM Corporation Accelerated Accelerators Kepler Pascal Volta CUDA 5.5 – 7.0 CUDA 8 CUDA 9 Unified Memory Full GPU Paging Cache Coherent Kepler Pascal Volta 1.5TF 16GB @ 1TB/s > 7.0TF 12GB @ 288GB/s 16GB @ 1.2TB/s SXM2 SXM2 PCIe NVLink 1.0 >40+40 GB/s NVLink 2.0 POWER8 On Demand Paging >75+75 GB/s Coherent POWER8+ POWER9 Buffered Memory 2014-2015 2016 2017 © 2015 IBM Corporation 7 Memory Hierarchy and data locality – single node processor core Control units L2 registers cache L3 Main L1 cache • Memory hierarchy cache Memory tries to exploit DDR locality Processor core • CPU: low latency Control units L2 design registers cache L1 cache Accelerator • Data transfers to accelerator are very Control units costly L2 Main Memory registers • Accelerator: high cache GDDR/HBM latency and high L1 cache bandwidth © 2015 IBM Corporation 8 . Parallel Computing: architecture overview © 2015 IBM Corporation 9 Main architecture trends and characteristics . More and more cores (CPU and GPU) per node with Simultaneous Multiple Threading (up to 8 on IBM Power) IBM P8 cores 12 cores Intel Broadwell AMD 4 GHz 8-18 Cores ~3GHz 16-24 core/MCM 128-bit FPU 256-bit FPU 256-bit FPU . Accelerator integration with unified memory hierarchic performance requires data locality . Vector floating point units and SIMD (Single Instruction Multiple Data) operations => Performance requires application vectorization (both operations and data) . Multiple levels of parallelism © 2015 IBM Corporation 10 Parallel Computing: architecture overview P P P P Uniform Memory access (UMA): Each processor/processes has uniform access to memory Memory BUS Shared Memory programming model Memory Cache Coherent Uniform Memory access P P P P Internal P P P P (ccNUMA): buses Time for memory access depends on data Memory BUS Memory BUS location. Local access is faster Shared Memory programming model Memory Memory PCIe Heterogeneous/Hybrid accelerated Accelerator Accelerator processor High performance local High performance local Each accelerator has it’s own local memory memory memory and address space (changing) GPU, FPGA, MIC GPU, FPGA, MIC Hybrid programing model © 2015 IBM Corporation 11 HPC cluster Distributed Memory : Each node has it’s own local memory. Must do message passing to exchange data between nodes (most popular approach is MPI) Cluster Architecture P P P P Internal P P P P P P P P Internal P P P P buses buses Memory BUS Memory BUS Memory BUS Memory BUS Memory Memory Memory Memory PCIe PCIe Accelerator Accelerator Accelerator Accelerator Network © 2015 IBM Corporation Programming Models and Languages Accelerator Address Space CUDA, OpenCL Process/Thread Address Space Accelerator Thread OpenMP, OpenACC .. .. .. .. .. .. .. .. .. Message passing Shared Memory PGAS MPI pThreads, OpenMP, OpenACC,Java UPC, CAF, X10 . Computation is performed in multiple . A datum in one place may reference a datum places. in another place. A place contains data that can be . Data-structures (e.g. arrays) may be operated on remotely. distributed across many places. Data lives in the place it was created, . Places may have different computational for its lifetime. properties 12 © 2015 IBM Corporation Where Does Performance Come From? . Computer Architecture – Instruction issue rate Not anymore distributed and shared . Execution pipelining memory paradigms . Reservation stations . Branch prediction . Cache & memory management Node – Parallelism Socket . Parallelism – number of operations Chip per cycle per processor Core Instruction level parallelism (ILP) Vector processing Thread . Parallelism – number of threads per Register/SIMD core . Parallelism – number of cores per Multiple instruction pipelines processor (SMT) . Parallelism – number of processors per node . Parallelism – number of accelerator per node . Parallelism – number of nodes in a Need to optimize for all levels! system . Device Technology – Memory capacity and access time – Communications bandwidth and latency – Logic switching speed and device density 13 © 2015 IBM Corporation 14 HPC Programming models & languages Multicore, Manycore, Standard ?? C/C++/Fortran OpenMP accelerator, large scale Python, R for Mapreduce/Spark data parallelism task parallelism Shared & Distributed UPC, CAF, ARMCI/Global Arrays, Memory CUDA, OpenCL, OpenACC,CILK, HMPP, StarSc, X10, Chapel, Fortress, Sisal, ... OPenMP, TBB, pTheads, MparReduce… Distributed Memory Many models & libraries Vector Units: SIMD -> MPI Shared Memory Single Memory C++, ADA, perl, Tcl, XML... High level: Fortran, LISP, COBOL, C, ... assembler 1950 1954 1980 1990 1995 2000 2010 2016 … © 2015 IBM Corporation Programming languages & Programming models 15 APGAS project review Different ways to program and Accelerate Applications Applications Libraries Compiler Specific Directives Programming OpenMP/OpnenACC/… Languages Easy to use Easy to use Less portable Most Performance Portable code Optimal performance Is there an existing Can I easily add Is it performance library that can do directives to help critical ? what I want ? the compiler ? © 2015 IBM Corporation 17 Programming languages . 2 main types languages – Compiled: C, C++, Fortran, ADA… . Compilers: GCC, CLANG/LLVM, IBM XL, INTEL, NVIDIA PGI, PathScale, Visual C/C++ – Interpreted: Python, java, R, Ruby, perl,… . Many programming models – Shared memory . Pthreads APIs, OpenMP/OpenACC directives for C/C++/Fortran, TBB-Thread Building Blocks, CILK - Lightweight threads embedded into C, java threads, … – Accelerator . OpenMP4.x, OpenACC directives for C/C++/Fortran, CUDA& OpenCL APIs, libspe, ATI,, StarPU (INRIA), SequenceL, VHDL for FPGA, … – Distributed memory . MPI, Sockets, PGAS (UPC, CAF…), … Strong focus and development effort for OpenMP (IBM, NVIDIA, INTEL) © 2015 IBM Corporation 18 High Performance Programming overview . For a programmer language should not be the barrier. The critical points are – To identify and extract parallelism – the programming model, as from language to language mainly syntax changes Scalar computing THINK PARALLEL Data Vector Aware computing Computing Express Express Data Identify Parallelism Optimize Parallelism Locality critical © 2015 IBM Corporation 19 Before chosing a programming model & languages 1. What paralellism you could extract? 2. What are the characteristics of your application? 3. Which curve are you on? 4. What are the current performances? Performance # of Cores 5. What performance do you need? 6. When do you want to reach your target? 7. What’s the life span of your application, versus hardware life span? 8. What are your technical resources and skills? © 2015 IBM Corporation 20 Programming models for HPC . The challenge is to efficiently map a problem to the architecture – Address parallel paradigms for large futures systems (vector, threading, data-parallel and transfers, message-passing, accelerator…) – Address scalability – Take advantage of all computational resources – Support well performance programming – Take advantage of advances in compiler – Interoperable with existing languages – Guaranty portability . For a programmer language should not be the barrier. The critical point is the programming model supported, other criteria: portability, simplicity, efficiency, readability . Main languages for traditional HPC applications: – C/C++, Fortran, Python, R . Languages evolution: more parallelism and hybrid computing feature (C++17, OpenMP 4.5, OpenACC 3.0, UPC, CAF …) © 2015 IBM Corporation Beyond multi-core and parallelism . The problem is not multi-node, multi-core, many-core, … But . The problem is in the application programmer’s head – Do I have parallelism? – What is the right programming model for concurrency or/and heterogeneity, efficiency, readability, manageability, …? – Address clusters, SMPs, multi-cores, accelerators… . Common trends – more and more processes and threads – Data centric . How to estimate the development cost and impacts for . entrance . exit 21 © 2015 IBM Corporation 22 Vectorization overview each current and future core has vector units © 2015 IBM Corporation SIMD – Single Instruction Multiple Data . Scalar Processing . SIMD Processing – Traditional mode – One operation produces multiple results – One operation produces one result • parallel vector operations • applies the same operation in parallel on a number of data items packed into a 128-512-bit vector (2-8 DP operation per cycle) • Without vector operation peak performance must divide by vector lenght • There are many different versions of SIMD extensions • SSE, AVX, AVX2, AVX-512, Altivec, VMX 23 © 2015 IBM Corporation 24 Vectorization example - Single DAXPY : A*X Plus Y . 3 ways to enable vector operations: compiler, library and Instrinsic APIs Using the compiler (« portable
Recommended publications
  • Swri IR&D Program 2016
    Internal Research and Development 2016 The SwRI IR&D Program exists to broaden the Institute's technology base and to encourage staff professional growth. Internal funding of research enables the Institute to advance knowledge, increase its technical capabilities, and expand its reputation as a leader in science and technology. The program also allows Institute engineers and scientists to continually grow in their technical fields by providing freedom to explore innovative and unproven concepts without contractual restrictions and expectations. Space Science Materials Research & Structural Mechanics Intelligent Systems, Advanced Computer & Electronic Technology, & Automation Engines, Fuels, Lubricants, & Vehicle Systems Geology & Nuclear Waste Management Fluid & Machinery Dynamics Electronic Systems & Instrumentation Chemistry & Chemical Engineering Copyright© 2017 by Southwest Research Institute. All rights reserved under U.S. Copyright Law and International Conventions. No part of this publication may be reproduced in any form or by any means, electronic or mechanical, including photocopying, without permission in writing from the publisher. All inquiries should be addressed to Communications Department, Southwest Research Institute, P.O. Drawer 28510, San Antonio, Texas 78228-0510, [email protected], fax (210) 522-3547. 2016 IR&D | IR&D Home SwRI IR&D 2016 – Space Science Capability Development and Demonstration for Next-Generation Suborbital Research, 15-R8115 Scaling Kinetic Inductance Detectors, 15-R8311 Capability Development of
    [Show full text]
  • A Functional Approach to Finding Answer Sets
    A Functional Approach to Finding Answer Sets Bryant Nelson, Josh Archer, and Nelson Rushton Texas Tech Dept. of Computer Science (bryant.nelson | josh.archer | nelson.rushton) @ ttu.edu Texas Tech University Department of Computer Science Box 43104 Lubbock, TX 79409-3104 we have been convinced of Hughes’ hypothesis. We set out to further test this Submitted to FCS 2013 hypothesis by encoding an Answer Set Solver in SequenceL. We hypothesize that Keywords: Functional Programming, a functional language would allow the Answer Set Prolog, Logic Programming programmer to implement an Answer Set Solver more quickly than a procedural Abstract: A naive answer set solver was language, and that the resulting program implemented in the functional programming would be more readable and language SequenceL, and its performance understandable. compared to mainstream solvers on a set of standard benchmark problems. Finding answer sets is an NP-Complete Implementation was very rapid (25 person problem, and over the last few decades a hours) and the resulting program only 400 few solvers have been developed that lines of code. Nonetheless, the algorithm tackle this problem. The current juggernauts was tractable and obtained parallel in this area are the DLV, CLASP, and speedups. Performance, though not SMODELS solvers, all of which are very pathologically poor, was considerably efficient. The problem would seem to benefit slower (around 20x) than that of mainstream from a parallelized solver, yet CLASP only solvers (CLASP, Smodels, and DLV) on all has a branch (CLASPar) dedicated to this, but one benchmark problem. and DLV has been “experimenting” with parallelization in their grounder [DLVPar].
    [Show full text]
  • Bryant Nelson
    Bryant K. Nelson, PhD (940) 902-5460 [email protected] bethune-bryant bryant-nelson Education: Doctor of Philosophy in Computer Science, Texas Tech University 2016 “Automatic Distributed Programming Using SequenceL” Bachelor of Science in Computer Science, Texas Tech University 2012 Bachelor of Science in Mathematics, Texas Tech University 2012 Professional Experience: Texas Multicore Technologies Inc., Software Development Engineer Jul 2012- Present Austin, Texas o Develop and extend the programming language, SequenceL, including work on the compiler, interpreter, debugger and IDE. Develop and maintain the primary libraries shipped with SequenceL. o Work with companies porting existing projects to SequenceL to make use of multicore hardware. o Speak at technical conferences and briefings. Design and lead technical workshops and training. o Primary Languages: SequenceL, C++, C#, Haskell, Bash, make, Python o Primary Tools: Jenkins, Git, Jira, Cross-OS Development (Linux/Mac/Windows) Tyler Technologies Inc., Software Developer Mar 2011- Aug 2012 Lubbock, Texas o As part of the Public Safety Team I worked on projects such as, Computer Aided Dispatch, Unit Mapping and Control, Records Management, Mobile Unit Interfaces, and Statistical Crime Analysis. o Primary Languages: Visual Basic 6, C# .NET, SQL Texas Tech University, Computer Science Undergraduate Research Assistant Aug 2010-Jul 2012 Lubbock, Texas o Worked closely with NASA improving and rewriting flight code in a more efficient new language, SequenceL. SequenceL is a declarative, automatic-parallelizing, high-level language. Texas Tech University, Computer Science Supplemental Instructor Aug 2009 - May 2010 Lubbock, Texas o Taught a Supplemental Computer Science class multiple times a week intended to supplement the information given by the professor, and to assist students in developing good study habits.
    [Show full text]
  • Designing Interdisciplinary Approaches to Problem Solving Into Computer Languages
    Designing Interdisciplinary Approaches to Problem Solving Into Computer Languages by Daniel E. Cooke (1), Vladik Kreinovich (2), and Joseph E. Urban (3) (1) Computer Science Department Texas Tech University; (2) Computer Science Department University of Texas at El Paso; (3) Computer Science and Engineering Department Arizona State University Abstract Many interdisciplinary design efforts require the involvement of computer scientists because of the complexity of the problem solving tools available for the projects. This paper demonstrates how appropriate language design can place high level languages in the hands of scientists and engineers, thus providing a more automated approach to problem solving that may reduce the amount of computer scientist involvement. The language SequenceL serves as an example of this approach. 1.0 Introduction There is an ongoing discussion at the highest levels of the United States government concerning "data morgues." The concern has to do with the current hardware capabilities that permit the acquisition and storage of vast amounts of data and the inability of scientists armed with current software technology to process and analyze the data. The current problem actually demonstrates that advances in computer software have not kept pace with advances in computer hardware. If corresponding software advances can be made, the data may be found to be comatose, rather than dead. Among other root problems, the comatose data is symptomatic of the fact that currently available software technology is not based upon abstractions that appropriately simplify approaches to complex problems and data sets, especially when the data sets are distributed among multiple processing elements. If large data sets containing, e.g., telemetry data, are to be analyzed, then exploratory or data mining programs must be written.
    [Show full text]
  • Comparative Programming Languages CM20253
    We have briefly covered many aspects of language design And there are many more factors we could talk about in making choices of language The End There are many languages out there, both general purpose and specialist And there are many more factors we could talk about in making choices of language The End There are many languages out there, both general purpose and specialist We have briefly covered many aspects of language design The End There are many languages out there, both general purpose and specialist We have briefly covered many aspects of language design And there are many more factors we could talk about in making choices of language Often a single project can use several languages, each suited to its part of the project And then the interopability of languages becomes important For example, can you easily join together code written in Java and C? The End Or languages And then the interopability of languages becomes important For example, can you easily join together code written in Java and C? The End Or languages Often a single project can use several languages, each suited to its part of the project For example, can you easily join together code written in Java and C? The End Or languages Often a single project can use several languages, each suited to its part of the project And then the interopability of languages becomes important The End Or languages Often a single project can use several languages, each suited to its part of the project And then the interopability of languages becomes important For example, can you easily
    [Show full text]
  • Your Software Faster, Better, Sooner
    The Complete Solution BETTER SOFTWARE FASTER Express problems in engineering terms without worrying how it will compute. Focus on problem solving, innovation, and getting it right the first time. Your Software Faster, AUTO-PARALLELIZING Applications self-parallelize, with no effort on the part of the user, and are Better, Sooner provably race-free. AGILE & TRANSPORTABLE Faster time to market and the ability to quickly explore/support new hardware architectures. Applications developed for today’s technology automatically perform on tomorrow’s. The Right Tools for the Multicore INDUSTRY STANDARDS Built on open industry standard C++ and OpenCL to work with existing design Programming Job flows, tools, and training. Designed to work in concert with other programming languages, legacy code, and libraries. © 2016 Texas Multicore Technologies, Inc. TexasMulticore.com Unleashing the Potential of Today’s Multicore Systems The right tools for the multicore pro- SequenceL Gets Rid of Decades of SequenceL not only automatically gramming job. The days of program- Programming Baggage. With Se- parallelizes the code, but is provably ming in a one-size-fits all language quenceL, the programming paradigm race free. Race conditions are the are quickly becoming a thing of the no longer mimics old single-step CPU largest quality challenge a parallelized past, especially for multicore and hardware. Similar in concept to SQL software development team faces. many-core systems. SequenceL is a for databases, SequenceL is more like When combined with the many other compact, powerful functional pro- writing formulas on a white board. In defect sources, test and debug be- gramming language and auto- SequenceL, you describe the desired comes a much larger effort than the parallelizing tool set that quickly and output in terms of the input, as func- code development itself.
    [Show full text]
  • Exploring the Current Landscape of Programming Paradigm and Models
    Cover Venkatesh Prasad Ranganath Story Researcher at Microsoft Research, India Exploring the Current Landscape of Programming Paradigm and Models Until few years ago, imperative programming of Programming Paradigms[44]” page assignments (e.g. i = i + 1). This is evident and object-oriented programming have at Wikipedia states that “None of the by the explicitly use and maintenance been two prevalent programming main programming paradigms have a of a counter variable i, dependence on paradigms in the industry. However, this precise, globally unanimous defi nition, let the length of nums, and direct access of has started to change over the past few alone an offi cial international standard.” elements of nums (via indexing syntax, years with the community exploring and Hence, instead of chasing defi nitions of i.e. nums[i]). slowly adopting functional programming. programming paradigms, let us try to In object-oriented style, the Similarly, there has been a renewed understand programming paradigms via program does not depend on the interest in exploring existing programming examples. internal details of list type. Instead, languages or creating new programming We shall start with three prevalent list type encapsulates its elements languages that support either tried-and- programming paradigms: imperative, (implementation details) and provides forgotten paradigms or a combination of object-oriented, and functional. an iterator abstraction (interface) to paradigms. Further, there has been a surge For this purpose, let us consider iterate over its elements. Consequently, in languages and programming systems to the following programs that identical in the program depends only on the support development of loosely coupled functionality but written in imperative, iterator abstraction.
    [Show full text]
  • Automatic Parallelization Tools: a Review Varsha
    ISSN XXXX XXXX © 2017 IJESC Research Article Volume 7 Issue No.3 Automatic Parallelization Tools: A Review Varsha. K.. R Department of CSE RV College of Engineering, Bangalore, India Abstract: With the advent of multi core architectures, the concept of parallel programming is not restricted anymore to super-computers and enterprise level applications alone. Programmers are always ready to take advantage of the available potential of programs to be parallelized. The objective of the paper is to review various tools that are available for performing automatic parallelization. Keywords: Multi-core, DOALL, Loop-level Parallelis m, concurrency, data-dependence, profiling I. INTRODUCTION III. EXISTING MODELS The rise of the concept of parallel or concurrent programming The general procedure for these tools is to take a sequential code has led to the development of multi-core architectures on a wider as input and generate parallel codes. Using the sequential code, a range. Programmers find parallel programs executing on dependency matrix or graph is implemented which stores multiple cores or processors to be more efficient than serial ones. dependency information that exists between consecutive But it is a tedious task to write parallel programs directly or to statements in a loop. The next step is to detect a code section convert all existing serial programs to parallel. This limitation which has potential parallelism. Finally the tools generate a can be eliminated by automating this procedure. Tools are parallel code by inserting parallel programming constructs/ developed which can automatically accept a serial code and directives to potential DOALL loops. A loop can be considered generate a parallel code by inserting suitable parallel constructs.
    [Show full text]
  • Automatic Concurrency in Sequencel
    Electronic Notes in Theoretical Computer Science URL httpwwwelsevier nl lo ca te ent cs vol ume htm l pages Automatic Concurrency in SequenceL Daniel E Co oke Department of Computer Science Texas Tech University Lubbock TX USA Vladik Kreinovich Department of Computer Science University of Texas at El Paso El Paso TX USA Abstract This pap er presents a programming language which we b elieve to be most appro priate for the automation of parallel data pro cessing esp ecially data pro cessing of concern to the oil industry and to the US Federal Agencies involved in the analysis of Satellite Telemetry Data Fo cus is placed up on ma jor language issues facing the developmentof the information p ower grid The pap er presents an example of the typ e of parallelism desired in the Grid To implement this parallelism in such a lan guage as Javawe need to sp ecify parallelism explicitly We show that if we rewrite the same solution in the high level language SequenceL then parallelism b ecomes implicit SequenceL seems therefore to be a go o d candidate for a Grid Oriented Language b ecause its abstraction relieves the problem solver of much of the burden normally required in development of parallel problem solutions The Need for New Language Abstractions Hardware improvements and the general spread of computing and computer applications have created opp ortunities for scientists and engineers to solve ever more complicated problems However there are concerns ab out whether scientists and engineers p ossess the software to ols necessary to solve
    [Show full text]
  • AUTOMATIC DISTRIBUTED PROGRAMMING USING SEQUENCEL by Bryant K
    AUTOMATIC DISTRIBUTED PROGRAMMING USING SEQUENCEL by Bryant K. Nelson, B.S. A Dissertation In COMPUTER SCIENCE Submitted to the Graduate Faculty of Texas Tech University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Submitted to: Dr. Nelson Rushton Chair of Committee Dr. Richard Watson Dr. Bradley Nemanich Dr. Yong Chen Dr. Mark Sheridan Dean of the Graduate School August, 2016 c Bryant K. Nelson, 2016 Texas Tech University, Bryant Nelson, August 2016 For Dr. Dan Cooke, in memoriam. ii Texas Tech University, Bryant Nelson, August 2016 ACKNOWLEDGMENTS There are many people that made getting to where I am today possible. While I can never truly express the extent of my gratitude for these people, here I would like to take some time to acknowledge them and their contributions. First, I would like to acknowledge Dr. Nelson Rushton, the chair of my committee, my academic advisor, and my friend. Four years ago Dr. Rushton took me on as a PhD student, fresh out of my bachelor's programs. Over these past four years he has been there to guide me and offer advice as both a mentor and a friend. From collaborating on papers and guiding me down research paths to helping me get my grouping tighter and draw faster. Without Dr. Rushton's guidance and insight, I doubt that I would have made it this far in my academic career. Second, I would like to acknowledge my friend and colleague, Josh Archer. Josh and I actually started our undergraduate degrees at the same time and went through the same program, dual majoring in Computer Science and Mathematics.
    [Show full text]
  • Mali-T880 Launch Slides
    ARM: Power-efficient Compute for HPC CERN OCP HPC Conference Darren Cepulis [email protected] 1 CONFIDENTIAL Over 12B ARM Based Chips Shipped in 2014 2013 2014 Growth 5.1bn 5.4bn 300m Mobile 2.9bn 4.1bn 1.2b Embedded 1.8bn 1.9bn 100m Enterprise Home 0.6bn 0.6bn - ARM CPU Core Unit Shipments 2 CONFIDENTIAL ARM Architecture: Licensing Overview 1000+ 1B overall smartphones 350+ sold in 2013 customers licenses 50B 121 new 200+ chips licenses sold Cortex-M licenses, shipped in 2013 over 160 companies 10B 620 licenses ARM based sold in past 50 ARM chips shipped in five years v8-A licenses 2013 Hundreds of optimized system-on-chip solutions 3 CONFIDENTIAL 3 Extensible Architecture for Heterogeneous Multi-core Solutions Up to 24 I/O Up to 4 Virtualized Interrupts Heterogeneous processors – CPU, GPU, DSP and coherent cores per accelerators interfaces for cluster GIC-500 accelerators and I/O Cortex CPU Cortex CPU 10-40 or CHI or CHI DSPDSP GbE PCIe PCIe DSP SATA master master Cortex®-A72 Cortex-A72 Cortex-A72 Cortex-A72 DPI DPI Crypto USB ACE Up to 12 Cortex CPU Cortex CPU AHB NIC-400 coherent or CHI or CHI master master clusters Cortex-A53 Cortex-A53 Cortex-A53 Cortex-A53 I/O Virtualisation CoreLink MMU-500 CoreLink™ CCN-512 Cache Coherent Network 1-32MB L3 cache Snoop Filter Memory Memory Memory Memory Network Interconnect Network Interconnect Controller Controller Controller Controller Integrated NIC-400 NIC-400 L3 cache DMC-520 DMC-520 DMC-520 DMC-520 x72 x72 x72 x72 Flash SRAM GPIO PCIe DDR4-3200 DDR4-3200 DDR4-3200 DDR4-3200 Up to Quad channel DDR3/4 x72 Peripheral address space 4 CONFIDENTIAL ARM in Datacenter/HPC Compute .
    [Show full text]
  • A Parallel Communication Architecture for The
    A PARALLEL COMMUNICATION ARCHITECTURE FOR THE LANGUAGE SEQUENCEL by JULIAN B. RUSSBACH, B.S. A THESIS IN COMPUTER SCIENCE Submitted to the Graduate Faculty of Texas Tech University in Partial Fulfillment of the Requirements for the Degree of MASTER OF SCIENCE Approved Chairperson ofthe Committee Accepted Uean ot the Uraduate School December, 2004 ACKNOWLEDGEMENTS I would like to thank my committee chair Dr. Per Andersen for his wisdom, patience, and insight as a person and computer scientist; Dr. Nelson Rushton for his numerous ideas and contributions to SequenceL grammar and semantics, the inception of the token ring for dynamic load balancing SequenceL execution, and his contribution to the cluster; and Dr. Daniel E. Cooke for his enthusiasm, keen eye, cluster provisions, and the opportunity to work on a great research team. Thanks also to Chris G. Fielder for help with cluster assembly and troubleshooting; Chris McClimmans for cluster maintenance suggestions; and Dr. Philip Smith for use of the Texas Tech HPCC computers and valuable MPI lessons. A special thanks goes to my girlfriend Radoslava for her tolerance of me through a year of work. TABLE OF CONTENTS ACKNOWLEDGEMENTS ii ABSTRACT v LIST OF FIGURES vi CHAPTER I. INTRODUCTION 1 I. I Document Overview 3 1.2 Introduction to the Language SequenceL 4 1.2.1 What is SequenceL? 4 1.2.2 Consume-Simplify-Produce 8 1.2.3 Normalize-Transpose-Distribute 10 1.3 Other Parallel Languages 12 1.4 SequenceL Implementations 14 II. LITERATURE REVIEW 19 2.1.1 Load Balancing 19 2.2 Token rings 23 2.2.1 IBM's Token Ring 23 2.2.2 "token rings" 26 III.
    [Show full text]