DOCSLIB.ORG

Synchronized MIMD Computing Bradley C. Kuszmaul

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Synchronized MIMD Computing Bradley C. Kuszmaul Synchronized MIMD Computing by Bradley C. Kuszmaul SB mathematics Massachusetts Institute of Technology SB computer science and engineering Massachusetts Institute of Technology SM electrical engineering and computer science Massachusetts Institute of Technology Submitted to the Department of Electrical Engineering and Computer Science in partial ful®llment of the requirements for the degree of Doctor of Philosophy at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY May 1994 c Massachusetts Institute of Technology 1994. All rights reserved. : :: : : :: : :: : :: : : :: : :: : :: : : :: : :: : :: : : :: : :: : :: : : :: : :: : :: : : :: : :: : : :: : :: : :: : : :: Author : Department of Electrical Engineering and Computer Science May 22, 1994 :: : :: : : :: : :: : :: : : :: : :: : :: : : :: : :: : :: : : :: : :: : :: : : :: : :: : :: : : :: : :: : : :: : :: Certi®ed by :: Charles E. Leiserson Professor of Computer Science and Engineering Thesis Supervisor :: : :: : :: : : :: : :: : :: : : :: : :: : :: : : :: : :: : :: : : :: : :: : :: : : :: : :: : :: : : :: : :: : : :: Accepted by : F. R. Morgenthaler Chair, Department Committee on Graduate Students 1 2 Synchronized MIMD Computing by Bradley C. Kuszmaul Submitted to the Department of Electrical Engineering and Computer Science on May 22, 1994, in partial ful®llment of the requirements for the degree of Doctor of Philosophy Abstract Fast global synchronization provides simple, ef®cient solutions to many of the system problems of parallel computing. It achieves this by providing composition of both performance and correctness. B If you understand the performance and meaning of parallel computations A and , then you B understand the performance and meaning of ªA; barrier; º. To demonstrate this thesis, this dissertation ®rst describes the architecture of the Connection Machine CM-5 supercomputer, a synchronized MIMD (multiple instruction stream, multiple data stream) computer for which I was a principal architect. The CM-5 was designed to run programs written in the data-parallel style by providing fast global synchronization. Fast global synchro- nization also helps solve many of the system problems for the CM-5, including clock distribution, diagnostics, and timesharing. Global barrier synchronization is used frequently in data-parallel programs to guarantee correct- ness, but the barriers are often viewed as a performance overheadthat should be removedif possible. By studying speci®c mechanisms for using the CM-5 data network ef®ciently, the second part of this dissertation shows that this view is incorrect. Interspersing barriers during message sending can dramatically improve performance of many important message patterns. Barriers are compared to two other mechanisms, bandwidth matching and managing injection order, for improving the performance of sending messages. The last part of this dissertation explores the bene®ts of global synchronization for MIMD-style programs, which are less well understood than data-parallel programs. To understand the pro- gramming issues, I engineered the StarTech parallel chess program. Computer chess is a resource- intensive irregular MIMD-style computation, providing a challenging scheduling problem. Global synchronization allows us to write a scheduler which runs unstructured computations ef®ciently and predictably. Given such a scheduler, the run time of a dynamic MIMD-style program on a W particular machine becomes simply a function of the critical path length C and the total work . I W P + C + empirically found that the StarTech program executes in time T 1 02 1 5 4 3 seconds, which, except for the constant-term of 4.3 seconds, is within a factor of 2.52 of optimal. Thesis Supervisor: Charles E. Leiserson Title: Professor of Computer Science and Engineering 3 4 Contents Contents 5 List of Figures 7 1 Introduction 9 2 The Network Architecture of the Connection Machine CM-5 23 2.1 The CM-5 Network Interface 25 2.2 The CM-5 Data Network 26 2.3 The CM-5 Control Network 32 2.4 The CM-5 Diagnostic Network 37 2.5 Synchronized MIMD Goals 41 2.6 CM-5 History 43 3 Mechanisms for Data Network Performance 45 3.1 Introduction 45 3.2 CM-5 Background 47 3.3 Timing on the CM-5 49 3.4 Using Barriers Can Improve Performance 50 3.5 Packets Should Be Reordered 53 3.6 Bandwidth Matching 55 3.7 Programming Rules of Thumb 57 4 TheStarTechMassivelyParallelChessProgram 59 4.1 Introduction 59 4.2 Negamax Search Without Pruning 62 4.3 Alpha-Beta Pruning 63 4.4 ScoutSearch 65 4.5 Jamboree Search 66 4.6 Multithreading with Active Messages 67 4.7 The StarTech Scheduler 70 5 The Performance of the StarTech Program 73 5.1 Introduction 73 5.2 Analysis of Best-Ordered Trees 76 5.3 Analysis of Worst-Ordered Game Trees 81 5.4 Jamboree Search on Real Chess Positions 84 5.5 Scheduling Parallel Computer Chess is Demanding 93 5 5.6 Performance of the StarTech Scheduler 96 5.7 Swamping 101 5.8 A Space-Time Tradeoff 109 6 Tuning the StarTech Program 113 6.1 Introduction 113 6.2 What is the Right Way to Measure Speedup of a Chess Program? 113 6.3 The Global Transposition Table 116 6.4 Improving the Transposition Table Effectiveness 120 6.5 Ef®ciency Heuristics for Jamboree Search 127 6.6 How Time is Spent in StarTech 128 7 Conclusions 133 Acknowledgments 145 Bibliography 149 Biographical Note 161 6 List of Figures 1-1 The organization of a SIMD computer. 11 1-2 The organization of a synchronized MIMD computer. 11 1-3 A fragment of data-parallel code. 14 1-4 Naive synchronized-MIMD code. 14 1-5 Optimized synchronized-MIMD code. 14 1-6 Exploiting the split-phase barrier via code transformations. 15 2-1 The organization of the Connection Machine CM-5. 24 2-2 A binary fat-tree. 26 2-3 The interconnection pattern of the CM-5 data network. 27 2-4 The format of messages in the data network. 29 2-5 The format of messages in the control network. 35 2-6 Steering a token down the diagnostic network. 40 3-1 A 64-node CM-5 data-network fat-tree showing all of the major arms and their bandwidths (in each direction). 47 3-2 The CM-5 operating system in¯ates time when the network is full 49 3-3 The effect of barriers between block transfers on the cyclic-shift pattern. 51 3-4 The total number of packets in the network headed to any given processor at any given time. 52 3-5 Big blocks also suffer from target collisions. 53 3-6 The effect on performance of interleaving messages. 54 3-7 The effect of bandwidth matching on permutations separated by barriers. 56 4-1 Algorithm negamax. 63 4-2 Practical pruning: White to move and win. 64 4-3 Algorithm absearch. 64 4-4 Algorithm scout. 65 4-5 Algorithm jamboree. 66 4-6 A search tree and the sequence of stack con®gurations for a serial implementation. 67 4-7 A sequence of activation trees for a parallel tree search. 68 4-8 Three possible ways to for a frame to terminate. 69 4-9 Busy processors keep working while the split-phase barrier completes. 72 5-1 The critical tree of a best-ordered uniform game-tree. 77 5-2 Numerical values for average available parallelism. 78 5-3 Partitioning the critical tree to solve for the number of vertices. 79 5-4 Performance metrics for worst-ordered game trees. 82 5-5 Jamboree search sometimes does more work than a serial algorithm. 85 7 5-6 Jamboree search sometimes does less work than a serial algorithm. 86 5-7 The data¯ow graph for Jamboree search. 87 5-8 Computing the estimated critical path length using timestamping. 88 5-9 The critical path of a computation that includes parallel-or. 88 5-10 The total work of each of 25 chess positions. 90 5-11 The critical path of each of 25 chess positions. 91 5-12 Serial run time versus work ef®ciency. 92 5-13 The ideal-parallelism pro®le of a typical chess position. 94 5-14 The sorted ideal-parallelism pro®le. 94 5-15 The processor utilization pro®le of a typical chess position. 95 5-16
Load more

Recommended publications
  • Parallel Prefix Sum (Scan) with CUDA
    Parallel Prefix Sum (Scan) with CUDA Mark Harris [email protected] April 2007 Document Change History Version Date Responsible Reason for Change February 14, Mark Harris Initial release 2007 April 2007 ii Abstract Parallel prefix sum, also known as parallel Scan, is a useful building block for many parallel algorithms including sorting and building data structures. In this document we introduce Scan and describe step-by-step how it can be implemented efficiently in NVIDIA CUDA. We start with a basic naïve algorithm and proceed through more advanced techniques to obtain best performance. We then explain how to scan arrays of arbitrary size that cannot be processed with a single block of threads. Month 2007 1 Parallel Prefix Sum (Scan) with CUDA Table of Contents Abstract.............................................................................................................. 1 Table of Contents............................................................................................... 2 Introduction....................................................................................................... 3 Inclusive and Exclusive Scan .........................................................................................................................3 Sequential Scan.................................................................................................................................................4 A Naïve Parallel Scan ......................................................................................... 4 A Work-Efficient
    [Show full text]
  • CSE373: Data Structures & Algorithms Lecture 26
    CSE373: Data Structures & Algorithms Lecture 26: Parallel Reductions, Maps, and Algorithm Analysis Aaron Bauer Winter 2014 Outline Done: • How to write a parallel algorithm with fork and join • Why using divide-and-conquer with lots of small tasks is best – Combines results in parallel – (Assuming library can handle “lots of small threads”) Now: • More examples of simple parallel programs that fit the “map” or “reduce” patterns • Teaser: Beyond maps and reductions • Asymptotic analysis for fork-join parallelism • Amdahl’s Law • Final exam and victory lap Winter 2014 CSE373: Data Structures & Algorithms 2 What else looks like this? • Saw summing an array went from O(n) sequential to O(log n) parallel (assuming a lot of processors and very large n!) – Exponential speed-up in theory (n / log n grows exponentially) + + + + + + + + + + + + + + + • Anything that can use results from two halves and merge them in O(1) time has the same property… Winter 2014 CSE373: Data Structures & Algorithms 3 Examples • Maximum or minimum element • Is there an element satisfying some property (e.g., is there a 17)? • Left-most element satisfying some property (e.g., first 17) – What should the recursive tasks return? – How should we merge the results? • Corners of a rectangle containing all points (a “bounding box”) • Counts, for example, number of strings that start with a vowel – This is just summing with a different base case – Many problems are! Winter 2014 CSE373: Data Structures & Algorithms 4 Reductions • Computations of this form are called reductions
    [Show full text]
  • C5ENPA1-DS, C-5E NETWORK PROCESSOR SILICON REVISION A1
    Freescale Semiconductor, Inc... SILICON REVISION A1 REVISION SILICON C-5e NETWORK PROCESSOR Sheet Data Rev 03 PRELIMINARY C5ENPA1-DS/D Freescale Semiconductor,Inc. F o r M o r G e o I n t f o o : r w m w a t w i o . f n r e O e n s c T a h l i e s . c P o r o m d u c t , Freescale Semiconductor, Inc... Freescale Semiconductor,Inc. F o r M o r G e o I n t f o o : r w m w a t w i o . f n r e O e n s c T a h l i e s . c P o r o m d u c t , Freescale Semiconductor, Inc... Freescale Semiconductor,Inc. Silicon RevisionA1 C-5e NetworkProcessor Data Sheet Rev 03 C5ENPA1-DS/D F o r M o r Preli G e o I n t f o o : r w m w a t w i o . f n r e O e n s c T a h l i e s . c P o r o m m d u c t , inary Freescale Semiconductor, Inc... Freescale Semiconductor,Inc. F o r M o r G e o I n t f o o : r w m w a t w i o . f n r e O e n s c T a h l i e s . c P o r o m d u c t , Freescale Semiconductor, Inc. C5ENPA1-DS/D Rev 03 CONTENTS .
    [Show full text]
  • CSE 613: Parallel Programming Lecture 2
    CSE 613: Parallel Programming Lecture 2 ( Analytical Modeling of Parallel Algorithms ) Rezaul A. Chowdhury Department of Computer Science SUNY Stony Brook Spring 2019 Parallel Execution Time & Overhead et al., et Edition Grama nd 2 Source: “Introduction to Parallel Computing”, to Parallel “Introduction Parallel running time on 푝 processing elements, 푇푃 = 푡푒푛푑 – 푡푠푡푎푟푡 , where, 푡푠푡푎푟푡 = starting time of the processing element that starts first 푡푒푛푑 = termination time of the processing element that finishes last Parallel Execution Time & Overhead et al., et Edition Grama nd 2 Source: “Introduction to Parallel Computing”, to Parallel “Introduction Sources of overhead ( w.r.t. serial execution ) ― Interprocess interaction ― Interact and communicate data ( e.g., intermediate results ) ― Idling ― Due to load imbalance, synchronization, presence of serial computation, etc. ― Excess computation ― Fastest serial algorithm may be difficult/impossible to parallelize Parallel Execution Time & Overhead et al., et Edition Grama nd 2 Source: “Introduction to Parallel Computing”, to Parallel “Introduction Overhead function or total parallel overhead, 푇푂 = 푝푇푝 – 푇 , where, 푝 = number of processing elements 푇 = time spent doing useful work ( often execution time of the fastest serial algorithm ) Speedup Let 푇푝 = running time using 푝 identical processing elements 푇1 Speedup, 푆푝 = 푇푝 Theoretically, 푆푝 ≤ 푝 ( why? ) Perfect or linear or ideal speedup if 푆푝 = 푝 Speedup Consider adding 푛 numbers using 푛 identical processing Edition nd elements. Serial runtime,
    [Show full text]
  • Design and Implementation of a Stateful Network Packet Processing
    610 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 25, NO. 1, FEBRUARY 2017 Design and Implementation of a Stateful Network Packet Processing Framework for GPUs Giorgos Vasiliadis, Lazaros Koromilas, Michalis Polychronakis, and Sotiris Ioannidis Abstract— Graphics processing units (GPUs) are a powerful and TCAMs, have greatly reduced both the cost and time platform for building the high-speed network traffic process- to develop network traffic processing systems, and have ing applications using low-cost hardware. The existing systems been successfully used in routers [4], [7] and network tap the massively parallel architecture of GPUs to speed up certain computationally intensive tasks, such as cryptographic intrusion detection systems [27], [34]. These systems offer a operations and pattern matching. However, they still suffer from scalable method of processing network packets in high-speed significant overheads due to critical-path operations that are still environments. However, implementations based on special- being carried out on the CPU, and redundant inter-device data purpose hardware are very difficult to extend and program, transfers. In this paper, we present GASPP, a programmable net- and prohibit them from being widely adopted by the industry. work traffic processing framework tailored to modern graphics processors. GASPP integrates optimized GPU-based implemen- In contrast, the emergence of commodity many-core tations of a broad range of operations commonly used in the architectures, such as multicore CPUs and modern graph- network traffic processing applications, including the first purely ics processors (GPUs) has proven to be a good solution GPU-based implementation of network flow tracking and TCP for accelerating many network applications, and has led stream reassembly.
    [Show full text]
  • A Review of Multicore Processors with Parallel Programming
    International Journal of Engineering Technology, Management and Applied Sciences www.ijetmas.com September 2015, Volume 3, Issue 9, ISSN 2349-4476 A Review of Multicore Processors with Parallel Programming Anchal Thakur Ravinder Thakur Research Scholar, CSE Department Assistant Professor, CSE L.R Institute of Engineering and Department Technology, Solan , India. L.R Institute of Engineering and Technology, Solan, India ABSTRACT When the computers first introduced in the market, they came with single processors which limited the performance and efficiency of the computers. The classic way of overcoming the performance issue was to use bigger processors for executing the data with higher speed. Big processor did improve the performance to certain extent but these processors consumed a lot of power which started over heating the internal circuits. To achieve the efficiency and the speed simultaneously the CPU architectures developed multicore processors units in which two or more processors were used to execute the task. The multicore technology offered better response-time while running big applications, better power management and faster execution time. Multicore processors also gave developer an opportunity to parallel programming to execute the task in parallel. These days parallel programming is used to execute a task by distributing it in smaller instructions and executing them on different cores. By using parallel programming the complex tasks that are carried out in a multicore environment can be executed with higher efficiency and performance. Keywords: Multicore Processing, Multicore Utilization, Parallel Processing. INTRODUCTION From the day computers have been invented a great importance has been given to its efficiency for executing the task.
    [Show full text]
  • Parallel Algorithms and Parallel Program Design
    Introduction to Parallel Algorithms and Parallel Program Design Parallel Computing CIS 410/510 Department of Computer and Information Science Lecture 12 – Introduction to Parallel Algorithms Methodological Design q Partition ❍ Task/data decomposition q Communication ❍ Task execution coordination q Agglomeration ❍ Evaluation of the structure q Mapping I. Foster, “Designing and Building ❍ Resource assignment Parallel Programs,” Addison-Wesley, 1995. Book is online, see webpage. Introduction to Parallel Computing, University of Oregon, IPCC Lecture 12 – Introduction to Parallel Algorithms 2 Partitioning q Partitioning stage is intended to expose opportunities for parallel execution q Focus on defining large number of small task to yield a fine-grained decomposition of the problem q A good partition divides into small pieces both the computational tasks associated with a problem and the data on which the tasks operates q Domain decomposition focuses on computation data q Functional decomposition focuses on computation tasks q Mixing domain/functional decomposition is possible Introduction to Parallel Computing, University of Oregon, IPCC Lecture 12 – Introduction to Parallel Algorithms 3 Domain and Functional Decomposition q Domain decomposition of 2D / 3D grid q Functional decomposition of a climate model Introduction to Parallel Computing, University of Oregon, IPCC Lecture 12 – Introduction to Parallel Algorithms 4 Partitioning Checklist q Does your partition define at least an order of magnitude more tasks than there are processors in your target computer? If not, may loose design flexibility. q Does your partition avoid redundant computation and storage requirements? If not, may not be scalable. q Are tasks of comparable size? If not, it may be hard to allocate each processor equal amounts of work.
    [Show full text]
  • Embedded Multi-Core Processing for Networking
    12 Embedded Multi-Core Processing for Networking Theofanis Orphanoudakis University of Peloponnese Tripoli, Greece [email protected] Stylianos Perissakis Intracom Telecom Athens, Greece [email protected] CONTENTS 12.1 Introduction ............................ 400 12.2 Overview of Proposed NPU Architectures ............ 403 12.2.1 Multi-Core Embedded Systems for Multi-Service Broadband Access and Multimedia Home Networks . 403 12.2.2 SoC Integration of Network Components and Examples of Commercial Access NPUs .............. 405 12.2.3 NPU Architectures for Core Network Nodes and High-Speed Networking and Switching ......... 407 12.3 Programmable Packet Processing Engines ............ 412 12.3.1 Parallelism ........................ 413 12.3.2 Multi-Threading Support ................ 418 12.3.3 Specialized Instruction Set Architectures ....... 421 12.4 Address Lookup and Packet Classification Engines ....... 422 12.4.1 Classification Techniques ................ 424 12.4.1.1 Trie-based Algorithms ............ 425 12.4.1.2 Hierarchical Intelligent Cuttings (HiCuts) . 425 12.4.2 Case Studies ....................... 426 12.5 Packet Buffering and Queue Management Engines ....... 431 399 400 Multi-Core Embedded Systems 12.5.1 Performance Issues ................... 433 12.5.1.1 External DRAMMemory Bottlenecks ... 433 12.5.1.2 Evaluation of Queue Management Functions: INTEL IXP1200 Case ................. 434 12.5.2 Design of Specialized Core for Implementation of Queue Management in Hardware ................ 435 12.5.2.1 Optimization Techniques .......... 439 12.5.2.2 Performance Evaluation of Hardware Queue Management Engine ............. 440 12.6 Scheduling Engines ......................... 442 12.6.1 Data Structures in Scheduling Architectures ..... 443 12.6.2 Task Scheduling ..................... 444 12.6.2.1 Load Balancing ................ 445 12.6.3 Traffic Scheduling ...................
    [Show full text]
  • Parallel Processing! 1! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! 2! Suggested Readings! •! Readings! –! H&P: Chapter 7! •! (Over Next 2 Weeks)!
    CSE 30321 – Lecture 23 – Introduction to Parallel Processing! 1! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! 2! Suggested Readings! •! Readings! –! H&P: Chapter 7! •! (Over next 2 weeks)! Lecture 23" Introduction to Parallel Processing! University of Notre Dame! University of Notre Dame! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! 3! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! 4! Processor components! Multicore processors and programming! Processor comparison! vs.! Goal: Explain and articulate why modern microprocessors now have more than one core andCSE how software 30321 must! adapt to accommodate the now prevalent multi- core approach to computing. " Introduction and Overview! Writing more ! efficient code! The right HW for the HLL code translation! right application! University of Notre Dame! University of Notre Dame! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! 6! Pipelining and “Parallelism”! ! Load! Mem! Reg! DM! Reg! ALU ! Instruction 1! Mem! Reg! DM! Reg! ALU ! Instruction 2! Mem! Reg! DM! Reg! ALU ! Instruction 3! Mem! Reg! DM! Reg! ALU ! Instruction 4! Mem! Reg! DM! Reg! ALU Time! Instructions execution overlaps (psuedo-parallel)" but instructions in program issued sequentially." University of Notre Dame! University of Notre Dame! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! Multiprocessing (Parallel) Machines! Flynn#s
    [Show full text]
  • A Review of Parallel Processing Approaches to Robot Kinematics and Jacobian
    Technical Report 10/97, University of Karlsruhe, Computer Science Department, ISSN 1432-7864 A Review of Parallel Processing Approaches to Robot Kinematics and Jacobian Dominik HENRICH, Joachim KARL und Heinz WÖRN Institute for Real-Time Computer Systems and Robotics University of Karlsruhe, D-76128 Karlsruhe, Germany e-mail: [email protected] Abstract Due to continuously increasing demands in the area of advanced robot control, it became necessary to speed up the computation. One way to reduce the computation time is to distribute the computation onto several processing units. In this survey we present different approaches to parallel computation of robot kinematics and Jacobian. Thereby, we discuss both the forward and the reverse problem. We introduce a classification scheme and classify the references by this scheme. Keywords: parallel processing, Jacobian, robot kinematics, robot control. 1 Introduction Due to continuously increasing demands in the area of advanced robot control, it became necessary to speed up the computation. Since it should be possible to control the motion of a robot manipulator in real-time, it is necessary to reduce the computation time to less than the cycle rate of the control loop. One way to reduce the computation time is to distribute the computation over several processing units. There are other overviews and reviews on parallel processing approaches to robotic problems. Earlier overviews include [Lee89] and [Graham89]. Lee takes a closer look at parallel approaches in [Lee91]. He tries to find common features in the different problems of kinematics, dynamics and Jacobian computation. The latest summary is from Zomaya et al. [Zomaya96].
    [Show full text]
  • A Survey on Parallel Multicore Computing: Performance & Improvement
    Advances in Science, Technology and Engineering Systems Journal Vol. 3, No. 3, 152-160 (2018) ASTESJ www.astesj.com ISSN: 2415-6698 A Survey on Parallel Multicore Computing: Performance & Improvement Ola Surakhi*, Mohammad Khanafseh, Sami Sarhan University of Jordan, King Abdullah II School for Information Technology, Computer Science Department, 11942, Jordan A R T I C L E I N F O A B S T R A C T Article history: Multicore processor combines two or more independent cores onto one integrated circuit. Received: 18 May, 2018 Although it offers a good performance in terms of the execution time, there are still many Accepted: 11 June, 2018 metrics such as number of cores, power, memory and more that effect on multicore Online: 26 June, 2018 performance and reduce it. This paper gives an overview about the evolution of the Keywords: multicore architecture with a comparison between single, Dual and Quad. Then we Distributed System summarized some of the recent related works implemented using multicore architecture and Dual core show the factors that have an effect on the performance of multicore parallel architecture Multicore based on their results. Finally, we covered some of the distributed parallel system concepts Quad core and present a comparison between them and the multiprocessor system characteristics. 1. Introduction [6]. The heterogenous cores have more than one type of cores, they are not identical, each core can handle different application. Multicore is one of the parallel computing architectures which The later has better performance in terms of less power consists of two or more individual units (cores) that read and write consumption as will be shown later [2].
    [Show full text]
  • Parallelizing Multiple Flow Accumulation Algorithm Using CUDA and Openacc
    International Journal of Geo-Information Article Parallelizing Multiple Flow Accumulation Algorithm using CUDA and OpenACC Natalija Stojanovic * and Dragan Stojanovic Faculty of Electronic Engineering, University of Nis, 18000 Nis, Serbia * Correspondence: [email protected] Received: 29 June 2019; Accepted: 30 August 2019; Published: 3 September 2019 Abstract: Watershed analysis, as a fundamental component of digital terrain analysis, is based on the Digital Elevation Model (DEM), which is a grid (raster) model of the Earth surface and topography. Watershed analysis consists of computationally and data intensive computing algorithms that need to be implemented by leveraging parallel and high-performance computing methods and techniques. In this paper, the Multiple Flow Direction (MFD) algorithm for watershed analysis is implemented and evaluated on multi-core Central Processing Units (CPU) and many-core Graphics Processing Units (GPU), which provides significant improvements in performance and energy usage. The implementation is based on NVIDIA CUDA (Compute Unified Device Architecture) implementation for GPU, as well as on OpenACC (Open ACCelerators), a parallel programming model, and a standard for parallel computing. Both phases of the MFD algorithm (i) iterative DEM preprocessing and (ii) iterative MFD algorithm, are parallelized and run over multi-core CPU and GPU. The evaluation of the proposed solutions is performed with respect to the execution time, energy consumption, and programming effort for algorithm parallelization for different sizes of input data. An experimental evaluation has shown not only the advantage of using OpenACC programming over CUDA programming in implementing the watershed analysis on a GPU in terms of performance, energy consumption, and programming effort, but also significant benefits in implementing it on the multi-core CPU.
    [Show full text]