DOCSLIB.ORG

A Comparison of Cpus, Gpus, Fpgas, and Massively Parallel Processor Arrays for Random Number Generation

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Comparison of Cpus, Gpus, Fpgas, and Massively Parallel Processor Arrays for Random Number Generation A Comparison of CPUs, GPUs, FPGAs, and Massively Parallel Processor Arrays for Random Number Generation David B. Thomas Lee Howes Wayne Luk Imperial College London Imperial College London Imperial College London [email protected] [email protected] [email protected] ABSTRACT offered by parallel processing systems is still increasing with The future of high-performance computing is likely to rely each generation, through the introduction of multi-core ver- on the ability to efficiently exploit huge amounts of paral- sions of conventional CPUs, and through “unconventional” lelism. One way of taking advantage of this parallelism is computing platforms, such as FPGAs (Field Programmable to formulate problems as “embarrassingly parallel” Monte- Gate Arrays), GPUs (Graphics Processor Units), and MP- Carlo simulations, which allow applications to achieve a lin- PAs (Massively Parallel Processor Arrays). ear speedup over multiple computational nodes, without re- One application domain where it is often possible to both quiring a super-linear increase in inter-node communication. easily describe applications, and to take advantage of par- However, such applications are reliant on a cheap supply allel computational power, is Monte-Carlo simulation. This of high quality random numbers, particularly for the three is one of a class of “embarrassingly parallel” application do- main maximum entropy distributions: uniform, used as a mains, where as the number of computational elements in- general source of randomness; Gaussian, for discrete-time creases, the communication required between elements does simulations; and exponential, for discrete-event simulations. not increase super-linearly. We know that Monte-Carlo ap- In this paper we look at four different types of platform: plications are able to scale (approximately) linearly across conventional multi-core CPUs (Intel Core2); GPUs (NVidia parallel architectures, but this doesn’t necessarily mean that GTX 200); FPGAs (Xilinx Virtex-5); and Massively Paral- the new generation of parallel architectures will be ideal for lel Processor Arrays (Ambric AM2000). For each platform Monte-Carlo; we also need to know that they can efficiently we determine the most appropriate algorithm for generat- support the types of computation found in such applica- ing each type of number, then calculate the peak generation tions. This includes standard building-blocks such as addi- rate and estimated power efficiency for each device. tion, multiplication, and division, but also requires Random Number Generators (RNGs). The importance of efficient RNGs is demonstrated by the Categories and Subject Descriptors enormous effort applied by researchers to traditional soft- B.8.2 [Hardware]: Performance and Reliability—Perfor- ware techniques, both to improve the efficiency of genera- mance Analysis and Design Aids tion techniques, and to modify simulations to reduce the number of random numbers consumed per run. Such is General Terms the cost of random numbers in software, that in the Gib- son method for simulating discrete-time biochemistry it is Algorithms, Design, Performance preferable to perform four floating-point operations (includ- ing a divide) just so that a previous random number can Keywords be re-used, rather than the simpler approach of generating a new random number. However, in an FPGA accelerated Monte-Carlo, Random Numbers, FPGA, GPU, MPPA simulation the economics change: it is actually cheaper to generate a new random number, with the added benefit that 1. INTRODUCTION the simulation code is easier to write and understand. The power of conventional super-scalar CPUs has steadily To understand how new parallel architectures can and increased for many decades, but we have now reached the should be used to accelerate Monte-Carlo, we need to prop- point where processor performance does not scale geomet- erly understand the costs of RNG primitives. In this pa- rically with passing generations. However, the performance per we consider how best to implement RNGs across four platforms, and then calculate the peak generation rate and power efficiency of each device. Our contributions are: An examination of random number generation on four Permission to make digital or hard copies of all or part of this work for • personal or classroom use is granted without fee provided that copies are different architectures: CPU, GPU, MPPA, and FPGA. not made or distributed for profit or commercial advantage and that copies Selection of RNGs for the uniform, Gaussian, and ex- bear this notice and the full citation on the first page. To copy otherwise, to • republish, to post on servers or to redistribute to lists, requires prior specific ponential distribution for each platform. permission and/or a fee. A comparison of the four platform’s potential for Monte- FPGA’09, February 22–24, 2009, Monterey, California, USA. • Copyright 2009 ACM 978-1-60558-410-2/09/02 ...$5.00. Carlo applications, both in terms of raw generation speed, and estimated power efficiency. tical randomness provided by TestU01, we also require that the empirical distribution is extremely accurate. To test the 2. REQUIREMENTS FOR RNGS marginal distribution, we require that the generator is able to pass a χ2 test with 216 equal probability buckets, for at Highly parallel architectures present challenges for RNG least 236 samples. designers due to the enormous quantities of random num- bers generated and consumed during each application run. No specific requirements are made on data-types: if a gen- To ensure that we compare solutions that meet a common erator can pass the empirical quality tests, then whatever standard, we use the following requirements, which all the data-type (i.e. fixed-point or floating-point) it produces is generators benchmarked in this paper adhere to. assumed to be acceptable. Deterministic and Pseudo-Random: In this paper we only consider deterministic generators. This class of gener- 3. OVERVIEW OF RNG METHODS ator uses a deterministic state transition function, operat- There are a huge variety of methods for generating random ing on a finite-size state, and hence will produce a pseudo- numbers, both uniform and non-uniform, thanks to a stream random sequence that repeats with a fixed period. The rep- of theoretical advances and practical improvements over the etition property is critical in many applications, as it al- last 60 years. Devroye’s book [6] on the subject provides lows applications to be repeatedly executed using the same detailed coverage of methods up to the mid 1980s; more re- sequence of numbers, allowing “surprising” simulation runs cent developments in uniform generation are surveyed by to be re-examined, or to use the same random stimulus to L’Ecuyer [11], and Gaussian generators by Thomas [25]. drive different models. We also require that generators are There appear to be no recent surveys of exponential RNGs, pseudo-random, i.e. they should be designed to produce but many of the best techniques for Gaussian generation can a sequence that looks as random as possible (see later re- also be applied to exponential numbers. quirements on statistical quality). These requirements rules out both True Random Number Generators (TRNG) which 3.1 Uniform Generation use some source of physical randomness; and Quasi-Random The purpose of uniform RNGs is to produce a sequence Number Generators (QRNG) which produce a sequence that that appears as random as possible. Each generator has covers some multi-dimensional space “evenly”, rather than a finite set of states S, a deterministic stateless transition pseudo-randomly. function t from state to state, and a mapping m from each Period: The period of each random sequence must be at state to an output value in the continuous range [0, 1]. least 2160, which allows the overall sequence to be split up 64 64 t : S S m : S [0, 1] (1) into 2 sub-streams of length 2 , with an extra “safety- 7→ 7→ 32 factor” of 2 . Larger periods are of course desirable, and On execution the generator is started in some state s0 S. are achievable in software due to the high-ratio of memory Repeated application of t produces an infinite sequence∈ of to processing elements, but in highly-parallel devices it is successor states s1,s2, ..., and also a sequence of pseudo- likely to be difficult to allocate the large amounts of mem- random samples x1,x2, ...: ory needed to hold the state of each generator per element. However, parallel devices should be able to substitute in- si = t(si−1) xi = m(si) (2) creased processing per output sample for a large state, and As t is deterministic and S is finite, the output sequence so maintain statistical quality. must eventually repeat itself. The period p of the generator Stream-splitting: When executing Monte-Carlo applica- is defined as the shortest cycle within the sequence: tions in parallel it is critical that each node has an indepen- min : i : si+p = si 1 p S (3) dent stream of random numbers, completely independent of p ∀ ≤ ≤ | | the numbers on all other nodes. To ensure this indepen- dence we require that it must be possible to partition the One of the measures of efficiency of an RNG is how close stream into (at least) 264 non-overlapping sub-sequences of p is to S. In practise S is a vector of w storage bits, so a w length (at least) 264. Alternatively, the generator must have period close to 2 is desirable. a period so large that if 264 random sub-sequences of length The difficulty when creating RNGs is to balance the com- 264 are chosen, then the probability of any pair overlapping peting concerns. On the one hand we want extremely long is less than 2−64. periods, and sequences that are statistically random looking (even over long sub-sequences); but on the other, we want Empirical statistical quality: It is difficult to prove any- the computational cost per generated number to be very low, thing about the theoretical randomness of generators, par- and the size of the RNG state should not be excessive.
Load more

Recommended publications
  • 2.5 Classification of Parallel Computers
    52 // Architectures 2.5 Classification of Parallel Computers 2.5 Classification of Parallel Computers 2.5.1 Granularity In parallel computing, granularity means the amount of computation in relation to communication or synchronisation Periods of computation are typically separated from periods of communication by synchronization events. • fine level (same operations with different data) ◦ vector processors ◦ instruction level parallelism ◦ fine-grain parallelism: – Relatively small amounts of computational work are done between communication events – Low computation to communication ratio – Facilitates load balancing 53 // Architectures 2.5 Classification of Parallel Computers – Implies high communication overhead and less opportunity for per- formance enhancement – If granularity is too fine it is possible that the overhead required for communications and synchronization between tasks takes longer than the computation. • operation level (different operations simultaneously) • problem level (independent subtasks) ◦ coarse-grain parallelism: – Relatively large amounts of computational work are done between communication/synchronization events – High computation to communication ratio – Implies more opportunity for performance increase – Harder to load balance efficiently 54 // Architectures 2.5 Classification of Parallel Computers 2.5.2 Hardware: Pipelining (was used in supercomputers, e.g. Cray-1) In N elements in pipeline and for 8 element L clock cycles =) for calculation it would take L + N cycles; without pipeline L ∗ N cycles Example of good code for pipelineing: §doi =1 ,k ¤ z ( i ) =x ( i ) +y ( i ) end do ¦ 55 // Architectures 2.5 Classification of Parallel Computers Vector processors, fast vector operations (operations on arrays). Previous example good also for vector processor (vector addition) , but, e.g. recursion – hard to optimise for vector processors Example: IntelMMX – simple vector processor.
    [Show full text]
  • A Massively-Parallel Mixed-Mode Computer Designed to Support
    This paper appeared in th International Parallel Processing Symposium Proc of nd Work shop on Heterogeneous Processing pages NewportBeach CA April Triton A MassivelyParallel MixedMo de Computer Designed to Supp ort High Level Languages Christian G Herter Thomas M Warschko Walter F Tichy and Michael Philippsen University of Karlsruhe Dept of Informatics Postfach D Karlsruhe Germany Mo dula Abstract Mo dula pronounced Mo dulastar is a small ex We present the architectureofTriton a scalable tension of Mo dula for massively parallel program mixedmode SIMDMIMD paral lel computer The ming The programming mo del of Mo dula incor novel features of Triton are p orates b oth data and control parallelism and allows hronous and asynchronous execution mixed sync Support for highlevel machineindependent pro Mo dula is problemorientedinthesensethatthe gramming languages programmer can cho ose the degree of parallelism and mix the control mo de SIMD or MIMDlike as need Fast SIMDMIMD mode switching ed bytheintended algorithm Parallelism maybe nested to arbitrary depth Pro cedures may b e called Special hardware for barrier synchronization of from sequential or parallel contexts and can them multiple process groups selves generate parallel activity without any restric tions Most Mo dula programs can b e translated into ecient co de for b oth SIMD and MIMD archi A selfrouting deadlockfreeperfect shue inter tectures connect with latency hiding Overview of language extensions The architecture is the outcomeofanintegrated de Mo dula extends Mo dula
    [Show full text]
  • Massively Parallel Computing with CUDA
    Massively Parallel Computing with CUDA Antonino Tumeo Politecnico di Milano 1 GPUs have evolved to the point where many real world applications are easily implemented on them and run significantly faster than on multi-core systems. Future computing architectures will be hybrid systems with parallel-core GPUs working in tandem with multi-core CPUs. Jack Dongarra Professor, University of Tennessee; Author of “Linpack” Why Use the GPU? • The GPU has evolved into a very flexible and powerful processor: • It’s programmable using high-level languages • It supports 32-bit and 64-bit floating point IEEE-754 precision • It offers lots of GFLOPS: • GPU in every PC and workstation What is behind such an Evolution? • The GPU is specialized for compute-intensive, highly parallel computation (exactly what graphics rendering is about) • So, more transistors can be devoted to data processing rather than data caching and flow control ALU ALU Control ALU ALU Cache DRAM DRAM CPU GPU • The fast-growing video game industry exerts strong economic pressure that forces constant innovation GPUs • Each NVIDIA GPU has 240 parallel cores NVIDIA GPU • Within each core 1.4 Billion Transistors • Floating point unit • Logic unit (add, sub, mul, madd) • Move, compare unit • Branch unit • Cores managed by thread manager • Thread manager can spawn and manage 12,000+ threads per core 1 Teraflop of processing power • Zero overhead thread switching Heterogeneous Computing Domains Graphics Massive Data GPU Parallelism (Parallel Computing) Instruction CPU Level (Sequential
    [Show full text]
  • CS 677: Parallel Programming for Many-Core Processors Lecture 1
    1 CS 677: Parallel Programming for Many-core Processors Lecture 1 Instructor: Philippos Mordohai Webpage: mordohai.github.io E-mail: [email protected] Objectives • Learn how to program massively parallel processors and achieve – High performance – Functionality and maintainability – Scalability across future generations • Acquire technical knowledge required to achieve the above goals – Principles and patterns of parallel programming – Processor architecture features and constraints – Programming API, tools and techniques 2 Important Points • This is an elective course. You chose to be here. • Expect to work and to be challenged. • If your programming background is weak, you will probably suffer. • This course will evolve to follow the rapid pace of progress in GPU programming. It is bound to always be a little behind… 3 Important Points II • At any point ask me WHY? • You can ask me anything about the course in class, during a break, in my office, by email. – If you think a homework is taking too long or is wrong. – If you can’t decide on a project. 4 Logistics • Class webpage: http://mordohai.github.io/classes/cs677_s20.html • Office hours: Tuesdays 5-6pm and by email • Evaluation: – Homework assignments (40%) – Quizzes (10%) – Midterm (15%) – Final project (35%) 5 Project • Pick topic BEFORE middle of the semester • I will suggest ideas and datasets, if you can’t decide • Deliverables: – Project proposal – Presentation in class – Poster in CS department event – Final report (around 8 pages) 6 Project Examples • k-means • Perceptron • Boosting – General – Face detector (group of 2) • Mean Shift • Normal estimation for 3D point clouds 7 More Ideas • Look for parallelizable problems in: – Image processing – Cryptanalysis – Graphics • GPU Gems – Nearest neighbor search 8 Even More… • Particle simulations • Financial analysis • MCMC • Games/puzzles 9 Resources • Textbook – Kirk & Hwu.
    [Show full text]
  • Introduction to High Performance Computing
    Introduction to High Performance Computing Gregory G. Howes Department of Physics and Astronomy University of Iowa Iowa High Performance Computing Summer School University of Iowa Iowa City, Iowa 20-22 May 2013 Thank you Ben Rogers Information Technology Services Glenn Johnson Information Technology Services Mary Grabe Information Technology Services Amir Bozorgzadeh Information Technology Services Mike Jenn Information Technology Services Preston Smith Purdue University and National Science Foundation Rosen Center for Advanced Computing, Purdue University Great Lakes Consortium for Petascale Computing This presentation borrows heavily from information freely available on the web by Ian Foster and Blaise Barney (see references) Outline • Introduction • Thinking in Parallel • Parallel Computer Architectures • Parallel Programming Models • References Introduction Disclaimer: High Performance Computing (HPC) is valuable to a variety of applications over a very wide range of fields. Many of my examples will come from the world of physics, but I will try to present them in a general sense Why Use Parallel Computing? • Single processor speeds are reaching their ultimate limits • Multi-core processors and multiple processors are the most promising paths to performance improvements Definition of a parallel computer: A set of independent processors that can work cooperatively to solve a problem. Introduction The March towards Petascale Computing • Computing performance is defined in terms of FLoating-point OPerations per Second (FLOPS) GigaFLOP 1 GF = 109 FLOPS TeraFLOP 1 TF = 1012 FLOPS PetaFLOP 1 PF = 1015 FLOPS • Petascale computing also refers to extremely large data sets PetaByte 1 PB = 1015 Bytes Introduction Performance improves by factor of ~10 every 4 years! Outline • Introduction • Thinking in Parallel • Parallel Computer Architectures • Parallel Programming Models • References Thinking in Parallel DEFINITION Concurrency: The property of a parallel algorithm that a number of operations can be performed by separate processors at the same time.
    [Show full text]
  • 1 Parallelism
    Lecture 23: Parallelism • Administration – Take QUIZ 17 over P&H 7.1-5, before 11:59pm today – Project: Cache Simulator, Due April 29, 2010 • Last Time – On chip communication – How I/O works • Today – Where do architectures exploit parallelism? – What are the implications for programming models? – What are the implications for communication? – … for caching? UTCS 352, Lecture 23 1 Parallelism • What type of parallelism do applications have? • How can computer architectures exploit this parallelism? UTCS 352, Lecture 23 2 1 Granularity of Parallelism • Fine grain instruction level parallelism • Fine grain data parallelism • Coarse grain (data center) parallelism • Multicore parallelism UTCS 352, Lecture 23 3 Fine grain instruction level parallelism • Fine grain instruction level parallelism – Pipelining – Multi-issue (dynamic scheduling & issue) – VLIW (static issue) – Very Long Instruction Word • Each VLIW instruction includes up to N RISC-like operations • Compiler groups up to N independent operations • Architecture issues fixed size VLIW instructions UTCS 352, Lecture 23 4 2 VLIW Example Theory Let N = 4 Theoretically 4 ops/cycle • 8 VLIW = 32 operations in practice In practice, • Compiler cannot fill slots • Memory stalls - load stall - UTCS 352, Lecture 23 5 Multithreading • Performing multiple threads together – Replicate registers, PC, etc. – Fast switching between threads • Fine-grain multithreading – Switch threads after each cycle – Interleave instruction execution – If one thread stalls, others are executed • Medium-grain
    [Show full text]
  • Embarrassingly Parallel
    Algorithms PART I: Embarrassingly Parallel HPC Spring 2017 Prof. Robert van Engelen Overview n Ideal parallelism n Master-worker paradigm n Processor farms n Examples ¨ Geometrical transformations of images ¨ Mandelbrot set ¨ Monte Carlo methods n Load balancing of independent tasks n Further reading 3/30/17 HPC 2 Ideal Parallelism n An ideal parallel computation can be immediately divided into completely independent parts ¨ “Embarrassingly parallel” ¨ “Naturally parallel” n No special techniques or algorithms required input P0 P1 P2 P3 result 3/30/17 HPC 3 Ideal Parallelism and the Master-Worker Paradigm n Ideally there is no communication ¨ Maximum speedup n Practical embarrassingly parallel applications have initial communication and (sometimes) a final communication ¨ Master-worker paradigm where master submits jobs to workers ¨ No communications between workers Send initial data P0 P1 P2 P3 Master Collect results 3/30/17 HPC 4 Parallel Tradeoffs n Embarrassingly parallel with perfect load balancing: tcomp = ts / P assuming P workers and sequential execution time ts n Master-worker paradigm gives speedup only if workers have to perform a reasonable amount of work ¨ Sequential time > total communication time + one workers’ time ts > tp = tcomm + tcomp -1 ¨ Speedup SP = ts / tP = P tcomp / (tcomm + tcomp) = P / (r + 1) where r = tcomp / tcomm ¨ Thus SP ® P when r ® ¥ n However, communication tcomm can be expensive ¨ Typically ts < tcomm for small tasks, that is, the time to send/recv data to the workers is more expensive than doing all
    [Show full text]
  • Core Processors
    UNIVERSITY OF CALIFORNIA Los Angeles Parallel Algorithms for Medical Informatics on Data-Parallel Many-Core Processors A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Computer Science by Maryam Moazeni 2013 © Copyright by Maryam Moazeni 2013 ABSTRACT OF THE DISSERTATION Parallel Algorithms for Medical Informatics on Data-Parallel Many-Core Processors by Maryam Moazeni Doctor of Philosophy in Computer Science University of California, Los Angeles, 2013 Professor Majid Sarrafzadeh, Chair The extensive use of medical monitoring devices has resulted in the generation of tremendous amounts of data. Storage, retrieval, and analysis of such data require platforms that can scale with data growth and adapt to the various behavior of the analysis and processing algorithms. In recent years, many-core processors and more specifically many-core Graphical Processing Units (GPUs) have become one of the most promising platforms for high performance processing of data, due to the massive parallel processing power they offer. However, many of the algorithms and data structures used in medical and bioinformatics systems do not follow a data-parallel programming paradigm, and hence cannot fully benefit from the parallel processing power of ii data-parallel many-core architectures. In this dissertation, we present three techniques to adapt several non-data parallel applications in different dwarfs to modern many-core GPUs. First, we present a load balancing technique to maximize parallelism in non-serial polyadic Dynamic Programming (DP), which is a family of dynamic programming algorithms with more non-uniform data access pattern. We show that a bottom-up approach to solving the DP problem exploits more parallelism and therefore yields higher performance.
    [Show full text]
  • Embarrassingly Parallel Jobs Are Not Embarrassingly Easy to Schedule on the Grid
    Embarrassingly Parallel Jobs Are Not Embarrassingly Easy to Schedule on the Grid Enis Afgan, Purushotham Bangalore Department of Computer and Information Sciences University of Alabama at Birmingham 1300 University Blvd., Birmingham AL 35294-1170 {afgane, puri}@cis.uab.edu Abstract range greatly from only a few instances to several hundred instances and each instance can execute from Embarrassingly parallel applications represent several seconds or minutes to many hours. The end an important workload in today's grid environments. result is speedup of application’s execution that is only Scheduling and execution of this class of applications limited by the number of resource available. is considered mostly a trivial and well-understood Meanwhile, grid computing has emerged as the process on homogeneous clusters. However, while upcoming distributed computational infrastructure that grid environments provide the necessary enables virtualization and aggregation of computational resources, associated resource geographically distributed compute resources into a heterogeneity represents a new challenge for efficient seamless compute platform [2]. Characterized by task execution for these types of applications across heterogeneous resources with a wide range of network multiple resources. This paper presents a set of speeds between participating sites, high network examples illustrating how execution characteristics of latency, site and resource instability and a sea of local individual tasks, and consequently a job, are affected management policies, it seems the grid presents as by the choice of task execution resources, task many problems as it solves. Nonetheless, from the invocation parameters, and task input data attributes. early days of the grid, EP class of applications has It is the aim of this work to highlight this relationship been categorized as the killer application for the grid between an application and an execution resource to [3].
    [Show full text]
  • Massively Parallel Computers: Why Not Prirallel Computers for the Masses?
    Maslsively Parallel Computers: Why Not Prwallel Computers for the Masses? Gordon Bell Abstract In 1989 I described the situation in high performance computers including several parallel architectures that During the 1980s the computers engineering and could deliver teraflop power by 1995, but with no price science research community generally ignored parallel constraint. I felt SIMDs and multicomputers could processing. With a focus on high performance computing achieve this goal. A shared memory multiprocessor embodied in the massive 1990s High Performance looked infeasible then. Traditional, multiple vector Computing and Communications (HPCC) program that processor supercomputers such as Crays would simply not has the short-term, teraflop peak performanc~:goal using a evolve to a teraflop until 2000. Here's what happened. network of thousands of computers, everyone with even passing interest in parallelism is involted with the 1. During the first half of 1992, NEC's four processor massive parallelism "gold rush". Funding-wise the SX3 is the fastest computer, delivering 90% of its situation is bright; applications-wise massit e parallelism peak 22 Glops for the Linpeak benchmark, and Cray's is microscopic. While there are several programming 16 processor YMP C90 has the greatest throughput. models, the mainline is data parallel Fortran. However, new algorithms are required, negating th~: decades of 2. The SIMD hardware approach of Thinking Machines progress in algorithms. Thus, utility will no doubt be the was abandoned because it was only suitable for a few, Achilles Heal of massive parallelism. very large scale problems, barely multiprogrammed, and uneconomical for workloads. It's unclear whether The Teraflop: 1992 large SIMDs are "generation" scalable, and they are clearly not "size" scalable.
    [Show full text]
  • Introduction to Parallel Computing
    INTRODUCTION TO PARALLEL COMPUTING Plamen Krastev Office: 38 Oxford, Room 117 Email: [email protected] FAS Research Computing Harvard University OBJECTIVES: To introduce you to the basic concepts and ideas in parallel computing To familiarize you with the major programming models in parallel computing To provide you with with guidance for designing efficient parallel programs 2 OUTLINE: Introduction to Parallel Computing / High Performance Computing (HPC) Concepts and terminology Parallel programming models Parallelizing your programs Parallel examples 3 What is High Performance Computing? Pravetz 82 and 8M, Bulgarian Apple clones Image credit: flickr 4 What is High Performance Computing? Pravetz 82 and 8M, Bulgarian Apple clones Image credit: flickr 4 What is High Performance Computing? Odyssey supercomputer is the major computational resource of FAS RC: • 2,140 nodes / 60,000 cores • 14 petabytes of storage 5 What is High Performance Computing? Odyssey supercomputer is the major computational resource of FAS RC: • 2,140 nodes / 60,000 cores • 14 petabytes of storage Using the world’s fastest and largest computers to solve large and complex problems. 5 Serial Computation: Traditionally software has been written for serial computations: To be run on a single computer having a single Central Processing Unit (CPU) A problem is broken into a discrete set of instructions Instructions are executed one after another Only one instruction can be executed at any moment in time 6 Parallel Computing: In the simplest sense, parallel
    [Show full text]
  • Parallel Computing Introduction Alessio Turchi
    Parallel Computing Introduction Alessio Turchi Slides thanks to: Tim Mattson (Intel) Sverre Jarp (CERN) Vincenzo Innocente (CERN) 1 Outline • High Performance computing: A hardware system view • Parallel Computing: Basic Concepts • The Fundamental patterns of parallel Computing 5 The birth of Supercomputing • The CRAY-1A: – 12.5-nanosecond clock, – 64 vector registers, – 1 million 64-bit words of high- speed memory. – Speed: – 160 MFLOPS vector peak speed – 110 MFLOPS Linpack 1000 (best effort) • Cray software … by 1978 – Cray Operating System (COS), – the first automatically vectorizing Fortran compiler (CFT), On July 11, 1977, the CRAY-1A, serial . – Cray Assembler Language (CAL) number 3, was delivered to NCAR. The were introduced. system cost was $8.86 million ($7.9 million plus $1 million for the disks). http://www.cisl.ucar.edu/computers/gallery/cray/cray1.jsp Peak GFLOPS The Era of the Vector Supercomputer Vector the of Era The Supercomputers original The 10 20 30 40 50 60 0 • • • • Required modest changes to software (vectorization) software to changes modest Required configuration memory shared an in processors Multiple software and hardware specialized highly built, Custom data of vectors on operated that mainframes Large Cray 2 (4), 1985 Vector Vector Cray YMP (8), 1989 Cray C916 (16), 1991 Cray T932 (32), 1996 Supercomputer Center The Cray C916/512 the at Pittsburgh The attack of the killer micros • The Caltech Cosmic Cube developed by Charles Seitz and Geoffrey Fox in1981 • 64 Intel 8086/8087 processors • 128kB of memory per processor • 6-dimensional hypercube network The cosmic cube, Charles Seitz Launched the Communications of the ACM, Vol 28, number 1 January 1985, p.
    [Show full text]