DOCSLIB.ORG

Parallelization of Tree-Recursive Algorithms on a SIMD Machine *

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Parallelization of Tree-Recursive Algorithms on a SIMD Machine * From: AAAI Technical Report SS-93-04. Compilation copyright © 1993, AAAI (www.aaai.org). All rights reserved. Parallelization of Tree-Recursive Algorithms on a SIMD Machine * Curt Powley, Richard E. Korf, and Chris Ferguson Computer Science Department University of California, Los Angeles Los Angeles, CA 90024 ABSTRACT The set of tree-recursive algorithms includes constraint satisfaction using backtracking, iterative-deepening The set of tree-recursive algorithms is large, search such as IDA*, depth-first branch-and-bound, including constraint satisfaction using back- two-player game minimax search, and most divide-and- tracking, iterative-deepening search such as conquer algorithms. IDA*, depth-first branch-and-bound, two- player game minimax search, and many 1.2 SIMD Machines divide-and-conquer algorithms. We describe a structured method for implementing such Two examples of single-instruction, multiple-data algorithms on SIMDmachines, and identify (SIMD) machines are the MasPar1 and the Connec- measures for determining if a tree-recursive tion Machine (CM)2. The MasPar is composed of up application is amenable or ill-suited for SIMD to 16,384 (16K) processors. The Connection Machine, parallelization. Using these ideas, we evaluate the computer used in this study, is composed of up to results from four testbeds. 65,536 (64K) one-bit processors. Every processor on a SIMDmachine must execute the same instruction at the same time, or no instruction 1 Introduction at all. This can make programming and verification of straightforward tasks easier, but can considerably 1.1 Tree-RecursiveAlgorithms complicate the programming of complex tasks, such as tree-traversal. Because of this programmingconstraint, A tree-recursive algorithm is one that traverses a tree SIMDmachines have largely been used for "data par- in executing multiple recursive calls. A simple example allel" applications. Traversing a large, irregular tree is this procedure for calculating Fibonacci (n), where that is dynamically generated does not fit this computa- is a non-negative integer. tional paradigm. On the other hand, tree traversal con- sists of performing the same computation of node ex- pansion and evaluation repeatedly. This suggests that fine-grain SIMDparallel computers might be appropri- Fib (n) ate for implementation of tree-recursive algorithms. if (n < 1) return (1) Someof this research was previously reported in [14], else return (Fib (n - 1) + Fib (n - [15], [16], [17], and [18]. *This work is supported in part by W. M. Keck Foun- 2 SIMD Tree Search (STS) dation grant W880615,NSF grants DIR-9024251and IRI- 9119825,NSF Biological Facilities grant BBS-8714206,the Defense Advanced Research Projects Agency under Con- Our basic SIMDTree Search algorithm, or STS, con- tract MDA903-87-C0663, Rockwell International, the Ad- sists of an initial distribution of nodesto processors, fol- vanced ComputingFacility of the Mathematics and Com- lowed by alternating phases of depth-first tree-traversal puter Science Division of ArgonneNational Laboratory, the (search) and load balancing [15]. University of Maryland Institute for AdvancedComputer Studies, the University of Minnesota/ArmyHigh Perfor- 1MasPar is a trademark of MasPar Computer manceComputing Research Center, the Massively Parallel Corporation. ComputerResearch Laboratory of Sandia National Labora- 2Connection Machine is a trademark of Thinking Ma- tories, and by Thinking MachinesCorporation. chines Corporation. 181 Initially, the tree consists of just the root node, located 3 Measures of Performance on a single active processor of the machine, and all the remaining processors are inactive. This processor ex- Our primary measures of performance are speedup and pands the root, generating its children, and assigns each efficiency. Speedup is the time that would be required child to a different processor. Each of these processors, by the most efficient serial program for the applica- in parallel, expands its node, generating more children tion, running on one processor of the SIMDmachine, that are assigned to different processors. This process divided by the time required by STS. Efficiency is sim- continues until there are no more free processors avail- ply speedup divided by the number of processors. Since able. we are interested in the factors that contribute to over- all efficiency, we decomposeefficiency into four compo- If the processor assigned to a node remains associated with that node after it is expanded, that processor will nents: raw speed ratio R, fraction of time working F, utilization U, and work ratio N. The product of these be wasted by not having any work to do during the subsequent depth-first search. To avoid this, when a four factors equals efficiency. node is expanded, its first child is assigned to the same The raw speed ratio is the ratio of the node gener- processor that held the parent, and only additional chil- ation rate of a single busy processor in the parallel dren are assigned to free processors. This guarantees STS algorithm, compared to the serial algorithm, and that once the initial distribution is completed, all pro- reflects the overhead of SIMDmachines in executing cessors will be assigned to nodes on the frontier of the conditional code. Next is the fraction of total time tree, and can participate in the depth-first search. that is devoted to working (searching), as opposed load balancing. A third factor is the processor utiliza- Oncethe initial distribution is completed, each proces- tion, which is the average fraction of processors that sor conducts a depth-first search of its assigned frontier are busy during search phases. Utilization reflects the node. Processors use a stack to represent the current extent to which processors finish their work and are idle path in the tree. Unfortunately, the trees generated by almost all tree-recursive problems of interest have irreg- while waiting to receive work in the next load balancing ular branching factors and depths, and some processors phase. The final factor is the ratio of total nodes gen- erated by the serial algorithm to total nodes generated will finish searching their subtrees long before others. by the parallel algorithm, or the work ratio. Depending Thus, load balancing is necessary to effectively use a on the application, this mayor may not be significant. large number of processors. On a MIMDmachine, these idle processors can get work from busy processors without interrupting the other 4 Constraint Satisfaction busy processors [19]. On a SIMD machine, however, since every processor must execute the same instruction Our simplest testbed for STS is backtracking for at the same time, or no instruction at all, in order to constraint-satisfaction problems. For example, the N- share work, all search activity must be temporarily sus- Queens problem is to place N queens on an N x N pended. Thus, when the number of active processors chessboard so that no two are on the same row, col- becomes small enough to make it worthwhile, search umn, or diagonal. In such applications, a path is cut stops and load balancing begins. off when a constraint is violated, or a solution found. The N-Queens problem can be solved by placing queens Two important problems are determining when to trig- one at a time. Whena constraint is violated, the search ger load balancing, and how to distribute the work. backtracks to the last untried legal move. STS uses a dynamic trigger [14] that automatically ad- justs itself to the problem size, and to different stages Using 16K processors on a CM2,we solved a 16-queens within a single problem. It is a greedy approach that problem with a speedup of 10,669, for an efficiency of maximizes the average rate of work over a search/load- 65%. This corresponds to a raw speed ratio of .750, a balance cycle. After triggering occurs, the work is re- fraction of time workingof 0.870, a utilization of 0.998, distributed during load balancing as follows. Each busy and a work ratio of 1.0. The work ratio is one because processor scans its stack to find a node with unexplored we found all solutions, making the search tree identical children [19]. Whenthere are more active processors in the serial and parallel cases. The total work done was than idle processors, a subset of the active processors 18.02 billion node generations in 5.2 hours, for a rate must be selected to export work. Criteria for choos- of 58 million node generations per minute. In contrast, ing exporting processors include: (1) largest estimated the node generation rate on a Hewlett Packard 9000 load [14], (2) randomselection [7], (3) selection of least- model 350 workstation was 2.1 million per minute. recently used [8], and (4) location of the processor’s node in the tree so that the tree is searched in more 5 Iterative-Deepening of a left-to-right fashion [20]. The effectiveness of the work-distribution method depends on the application. Another important algorithm in this class is iterative- deepening search, such as depth-first iterative- 182 deepening (DFID) and Iterative-Deepening A* (IDA*) Iterative-deepening is also used in these algorithms, and [9]. DFIDperforms a series of depth-first searches. A in fact originated in this setting [21]. Alpha and Beta path is cut off when the depth of the path exceeds a bounds produced by searching one subtree affect the depth threshold for the iteration. Thresholds for suc- amount of work necessary to search neighboring sub- ceeding iterations are increased so that when a solution trees. These local bounds must be propagated through- is found it is guaranteed to be of lowest cost. IDA* out the tree, and a parallel algorithm will almost cer- reduces the search space by adding to the path cost a tainly do more work than a serial algorithm. Thus, heuristic estimate of the cost to reach a goal node. the work ratio becomes especially important in these applications.
Load more

Recommended publications
  • Parallel Prefix Sum (Scan) with CUDA
    Parallel Prefix Sum (Scan) with CUDA Mark Harris [email protected] April 2007 Document Change History Version Date Responsible Reason for Change February 14, Mark Harris Initial release 2007 April 2007 ii Abstract Parallel prefix sum, also known as parallel Scan, is a useful building block for many parallel algorithms including sorting and building data structures. In this document we introduce Scan and describe step-by-step how it can be implemented efficiently in NVIDIA CUDA. We start with a basic naïve algorithm and proceed through more advanced techniques to obtain best performance. We then explain how to scan arrays of arbitrary size that cannot be processed with a single block of threads. Month 2007 1 Parallel Prefix Sum (Scan) with CUDA Table of Contents Abstract.............................................................................................................. 1 Table of Contents............................................................................................... 2 Introduction....................................................................................................... 3 Inclusive and Exclusive Scan .........................................................................................................................3 Sequential Scan.................................................................................................................................................4 A Naïve Parallel Scan ......................................................................................... 4 A Work-Efficient
    [Show full text]
  • Simulating Physics with Computers
    International Journal of Theoretical Physics, VoL 21, Nos. 6/7, 1982 Simulating Physics with Computers Richard P. Feynman Department of Physics, California Institute of Technology, Pasadena, California 91107 Received May 7, 1981 1. INTRODUCTION On the program it says this is a keynote speech--and I don't know what a keynote speech is. I do not intend in any way to suggest what should be in this meeting as a keynote of the subjects or anything like that. I have my own things to say and to talk about and there's no implication that anybody needs to talk about the same thing or anything like it. So what I want to talk about is what Mike Dertouzos suggested that nobody would talk about. I want to talk about the problem of simulating physics with computers and I mean that in a specific way which I am going to explain. The reason for doing this is something that I learned about from Ed Fredkin, and my entire interest in the subject has been inspired by him. It has to do with learning something about the possibilities of computers, and also something about possibilities in physics. If we suppose that we know all the physical laws perfectly, of course we don't have to pay any attention to computers. It's interesting anyway to entertain oneself with the idea that we've got something to learn about physical laws; and if I take a relaxed view here (after all I'm here and not at home) I'll admit that we don't understand everything.
    [Show full text]
  • Think in G Machines Corporation Connection
    THINK ING MACHINES CORPORATION CONNECTION MACHINE TECHNICAL SUMMARY The Connection Machine System Connection Machine Model CM-2 Technical Summary ................................................................. Version 6.0 November 1990 Thinking Machines Corporation Cambridge, Massachusetts First printing, November 1990 The information in this document is subject to change without notice and should not be construed as a commitment by Thinking Machines Corporation. Thinking Machines Corporation reserves the right to make changes to any products described herein to improve functioning or design. Although the information in this document has been reviewed and is believed to be reliable, Thinking Machines Corporation does not assume responsibility or liability for any errors that may appear in this document. Thinking Machines Corporation does not assume any liability arising from the application or use of any information or product described herein. Connection Machine® is a registered trademark of Thinking Machines Corporation. CM-1, CM-2, CM-2a, CM, and DataVault are trademarks of Thinking Machines Corporation. C*®is a registered trademark of Thinking Machines Corporation. Paris, *Lisp, and CM Fortran are trademarks of Thinking Machines Corporation. C/Paris, Lisp/Paris, and Fortran/Paris are trademarks of Thinking Machines Corporation. VAX, ULTRIX, and VAXBI are trademarks of Digital Equipment Corporation. Symbolics, Symbolics 3600, and Genera are trademarks of Symbolics, Inc. Sun, Sun-4, SunOS, and Sun Workstation are registered trademarks of Sun Microsystems, Inc. UNIX is a registered trademark of AT&T Bell Laboratories. The X Window System is a trademark of the Massachusetts Institute of Technology. StorageTek is a registered trademark of Storage Technology Corporation. Trinitron is a registered trademark of Sony Corporation.
    [Show full text]
  • CSE373: Data Structures & Algorithms Lecture 26
    CSE373: Data Structures & Algorithms Lecture 26: Parallel Reductions, Maps, and Algorithm Analysis Aaron Bauer Winter 2014 Outline Done: • How to write a parallel algorithm with fork and join • Why using divide-and-conquer with lots of small tasks is best – Combines results in parallel – (Assuming library can handle “lots of small threads”) Now: • More examples of simple parallel programs that fit the “map” or “reduce” patterns • Teaser: Beyond maps and reductions • Asymptotic analysis for fork-join parallelism • Amdahl’s Law • Final exam and victory lap Winter 2014 CSE373: Data Structures & Algorithms 2 What else looks like this? • Saw summing an array went from O(n) sequential to O(log n) parallel (assuming a lot of processors and very large n!) – Exponential speed-up in theory (n / log n grows exponentially) + + + + + + + + + + + + + + + • Anything that can use results from two halves and merge them in O(1) time has the same property… Winter 2014 CSE373: Data Structures & Algorithms 3 Examples • Maximum or minimum element • Is there an element satisfying some property (e.g., is there a 17)? • Left-most element satisfying some property (e.g., first 17) – What should the recursive tasks return? – How should we merge the results? • Corners of a rectangle containing all points (a “bounding box”) • Counts, for example, number of strings that start with a vowel – This is just summing with a different base case – Many problems are! Winter 2014 CSE373: Data Structures & Algorithms 4 Reductions • Computations of this form are called reductions
    [Show full text]
  • CSE 613: Parallel Programming Lecture 2
    CSE 613: Parallel Programming Lecture 2 ( Analytical Modeling of Parallel Algorithms ) Rezaul A. Chowdhury Department of Computer Science SUNY Stony Brook Spring 2019 Parallel Execution Time & Overhead et al., et Edition Grama nd 2 Source: “Introduction to Parallel Computing”, to Parallel “Introduction Parallel running time on 푝 processing elements, 푇푃 = 푡푒푛푑 – 푡푠푡푎푟푡 , where, 푡푠푡푎푟푡 = starting time of the processing element that starts first 푡푒푛푑 = termination time of the processing element that finishes last Parallel Execution Time & Overhead et al., et Edition Grama nd 2 Source: “Introduction to Parallel Computing”, to Parallel “Introduction Sources of overhead ( w.r.t. serial execution ) ― Interprocess interaction ― Interact and communicate data ( e.g., intermediate results ) ― Idling ― Due to load imbalance, synchronization, presence of serial computation, etc. ― Excess computation ― Fastest serial algorithm may be difficult/impossible to parallelize Parallel Execution Time & Overhead et al., et Edition Grama nd 2 Source: “Introduction to Parallel Computing”, to Parallel “Introduction Overhead function or total parallel overhead, 푇푂 = 푝푇푝 – 푇 , where, 푝 = number of processing elements 푇 = time spent doing useful work ( often execution time of the fastest serial algorithm ) Speedup Let 푇푝 = running time using 푝 identical processing elements 푇1 Speedup, 푆푝 = 푇푝 Theoretically, 푆푝 ≤ 푝 ( why? ) Perfect or linear or ideal speedup if 푆푝 = 푝 Speedup Consider adding 푛 numbers using 푛 identical processing Edition nd elements. Serial runtime,
    [Show full text]
  • A Review of Multicore Processors with Parallel Programming
    International Journal of Engineering Technology, Management and Applied Sciences www.ijetmas.com September 2015, Volume 3, Issue 9, ISSN 2349-4476 A Review of Multicore Processors with Parallel Programming Anchal Thakur Ravinder Thakur Research Scholar, CSE Department Assistant Professor, CSE L.R Institute of Engineering and Department Technology, Solan , India. L.R Institute of Engineering and Technology, Solan, India ABSTRACT When the computers first introduced in the market, they came with single processors which limited the performance and efficiency of the computers. The classic way of overcoming the performance issue was to use bigger processors for executing the data with higher speed. Big processor did improve the performance to certain extent but these processors consumed a lot of power which started over heating the internal circuits. To achieve the efficiency and the speed simultaneously the CPU architectures developed multicore processors units in which two or more processors were used to execute the task. The multicore technology offered better response-time while running big applications, better power management and faster execution time. Multicore processors also gave developer an opportunity to parallel programming to execute the task in parallel. These days parallel programming is used to execute a task by distributing it in smaller instructions and executing them on different cores. By using parallel programming the complex tasks that are carried out in a multicore environment can be executed with higher efficiency and performance. Keywords: Multicore Processing, Multicore Utilization, Parallel Processing. INTRODUCTION From the day computers have been invented a great importance has been given to its efficiency for executing the task.
    [Show full text]
  • Parallel Algorithms and Parallel Program Design
    Introduction to Parallel Algorithms and Parallel Program Design Parallel Computing CIS 410/510 Department of Computer and Information Science Lecture 12 – Introduction to Parallel Algorithms Methodological Design q Partition ❍ Task/data decomposition q Communication ❍ Task execution coordination q Agglomeration ❍ Evaluation of the structure q Mapping I. Foster, “Designing and Building ❍ Resource assignment Parallel Programs,” Addison-Wesley, 1995. Book is online, see webpage. Introduction to Parallel Computing, University of Oregon, IPCC Lecture 12 – Introduction to Parallel Algorithms 2 Partitioning q Partitioning stage is intended to expose opportunities for parallel execution q Focus on defining large number of small task to yield a fine-grained decomposition of the problem q A good partition divides into small pieces both the computational tasks associated with a problem and the data on which the tasks operates q Domain decomposition focuses on computation data q Functional decomposition focuses on computation tasks q Mixing domain/functional decomposition is possible Introduction to Parallel Computing, University of Oregon, IPCC Lecture 12 – Introduction to Parallel Algorithms 3 Domain and Functional Decomposition q Domain decomposition of 2D / 3D grid q Functional decomposition of a climate model Introduction to Parallel Computing, University of Oregon, IPCC Lecture 12 – Introduction to Parallel Algorithms 4 Partitioning Checklist q Does your partition define at least an order of magnitude more tasks than there are processors in your target computer? If not, may loose design flexibility. q Does your partition avoid redundant computation and storage requirements? If not, may not be scalable. q Are tasks of comparable size? If not, it may be hard to allocate each processor equal amounts of work.
    [Show full text]
  • Parallel Processing! 1! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! 2! Suggested Readings! •! Readings! –! H&P: Chapter 7! •! (Over Next 2 Weeks)!
    CSE 30321 – Lecture 23 – Introduction to Parallel Processing! 1! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! 2! Suggested Readings! •! Readings! –! H&P: Chapter 7! •! (Over next 2 weeks)! Lecture 23" Introduction to Parallel Processing! University of Notre Dame! University of Notre Dame! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! 3! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! 4! Processor components! Multicore processors and programming! Processor comparison! vs.! Goal: Explain and articulate why modern microprocessors now have more than one core andCSE how software 30321 must! adapt to accommodate the now prevalent multi- core approach to computing. " Introduction and Overview! Writing more ! efficient code! The right HW for the HLL code translation! right application! University of Notre Dame! University of Notre Dame! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! 6! Pipelining and “Parallelism”! ! Load! Mem! Reg! DM! Reg! ALU ! Instruction 1! Mem! Reg! DM! Reg! ALU ! Instruction 2! Mem! Reg! DM! Reg! ALU ! Instruction 3! Mem! Reg! DM! Reg! ALU ! Instruction 4! Mem! Reg! DM! Reg! ALU Time! Instructions execution overlaps (psuedo-parallel)" but instructions in program issued sequentially." University of Notre Dame! University of Notre Dame! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! CSE 30321 – Lecture 23 – Introduction to Parallel Processing! Multiprocessing (Parallel) Machines! Flynn#s
    [Show full text]
  • A Review of Parallel Processing Approaches to Robot Kinematics and Jacobian
    Technical Report 10/97, University of Karlsruhe, Computer Science Department, ISSN 1432-7864 A Review of Parallel Processing Approaches to Robot Kinematics and Jacobian Dominik HENRICH, Joachim KARL und Heinz WÖRN Institute for Real-Time Computer Systems and Robotics University of Karlsruhe, D-76128 Karlsruhe, Germany e-mail: [email protected] Abstract Due to continuously increasing demands in the area of advanced robot control, it became necessary to speed up the computation. One way to reduce the computation time is to distribute the computation onto several processing units. In this survey we present different approaches to parallel computation of robot kinematics and Jacobian. Thereby, we discuss both the forward and the reverse problem. We introduce a classification scheme and classify the references by this scheme. Keywords: parallel processing, Jacobian, robot kinematics, robot control. 1 Introduction Due to continuously increasing demands in the area of advanced robot control, it became necessary to speed up the computation. Since it should be possible to control the motion of a robot manipulator in real-time, it is necessary to reduce the computation time to less than the cycle rate of the control loop. One way to reduce the computation time is to distribute the computation over several processing units. There are other overviews and reviews on parallel processing approaches to robotic problems. Earlier overviews include [Lee89] and [Graham89]. Lee takes a closer look at parallel approaches in [Lee91]. He tries to find common features in the different problems of kinematics, dynamics and Jacobian computation. The latest summary is from Zomaya et al. [Zomaya96].
    [Show full text]
  • A Survey on Parallel Multicore Computing: Performance & Improvement
    Advances in Science, Technology and Engineering Systems Journal Vol. 3, No. 3, 152-160 (2018) ASTESJ www.astesj.com ISSN: 2415-6698 A Survey on Parallel Multicore Computing: Performance & Improvement Ola Surakhi*, Mohammad Khanafseh, Sami Sarhan University of Jordan, King Abdullah II School for Information Technology, Computer Science Department, 11942, Jordan A R T I C L E I N F O A B S T R A C T Article history: Multicore processor combines two or more independent cores onto one integrated circuit. Received: 18 May, 2018 Although it offers a good performance in terms of the execution time, there are still many Accepted: 11 June, 2018 metrics such as number of cores, power, memory and more that effect on multicore Online: 26 June, 2018 performance and reduce it. This paper gives an overview about the evolution of the Keywords: multicore architecture with a comparison between single, Dual and Quad. Then we Distributed System summarized some of the recent related works implemented using multicore architecture and Dual core show the factors that have an effect on the performance of multicore parallel architecture Multicore based on their results. Finally, we covered some of the distributed parallel system concepts Quad core and present a comparison between them and the multiprocessor system characteristics. 1. Introduction [6]. The heterogenous cores have more than one type of cores, they are not identical, each core can handle different application. Multicore is one of the parallel computing architectures which The later has better performance in terms of less power consists of two or more individual units (cores) that read and write consumption as will be shown later [2].
    [Show full text]
  • Parallelizing Multiple Flow Accumulation Algorithm Using CUDA and Openacc
    International Journal of Geo-Information Article Parallelizing Multiple Flow Accumulation Algorithm using CUDA and OpenACC Natalija Stojanovic * and Dragan Stojanovic Faculty of Electronic Engineering, University of Nis, 18000 Nis, Serbia * Correspondence: [email protected] Received: 29 June 2019; Accepted: 30 August 2019; Published: 3 September 2019 Abstract: Watershed analysis, as a fundamental component of digital terrain analysis, is based on the Digital Elevation Model (DEM), which is a grid (raster) model of the Earth surface and topography. Watershed analysis consists of computationally and data intensive computing algorithms that need to be implemented by leveraging parallel and high-performance computing methods and techniques. In this paper, the Multiple Flow Direction (MFD) algorithm for watershed analysis is implemented and evaluated on multi-core Central Processing Units (CPU) and many-core Graphics Processing Units (GPU), which provides significant improvements in performance and energy usage. The implementation is based on NVIDIA CUDA (Compute Unified Device Architecture) implementation for GPU, as well as on OpenACC (Open ACCelerators), a parallel programming model, and a standard for parallel computing. Both phases of the MFD algorithm (i) iterative DEM preprocessing and (ii) iterative MFD algorithm, are parallelized and run over multi-core CPU and GPU. The evaluation of the proposed solutions is performed with respect to the execution time, energy consumption, and programming effort for algorithm parallelization for different sizes of input data. An experimental evaluation has shown not only the advantage of using OpenACC programming over CUDA programming in implementing the watershed analysis on a GPU in terms of performance, energy consumption, and programming effort, but also significant benefits in implementing it on the multi-core CPU.
    [Show full text]
  • Chapter 3. Parallel Algorithm Design Methodology
    CSci 493.65 Parallel Computing Prof. Stewart Weiss Chapter 3 Parallel Algorithm Design Chapter 3 Parallel Algorithm Design Debugging is twice as hard as writing the code in the rst place. Therefore, if you write the code as cleverly as possible, you are, by denition, not smart enough to debug it. - Brian Kernighan [1] 3.1 Task/Channel Model The methodology used in this course is the same as that used in the Quinn textbook [2], which is the task/channel methodology described by Foster [3]. In this model, a parallel program is viewed as a collection of tasks that communicate by sending messages to each other through channels. Figure 3.1 shows a task/channel representation of a hypothetical parallel program. A task consists of an executable unit (think of it as a program), together with its local memory and a collection of I/O ports. The local memory contains program code and private data, e.e., the data to which it has exclusive access. Access to this memory is called a local data access. The only way that a task can send copies of its local data to other tasks is through its output ports, and conversely, it can only receive data from other tasks through its input ports. An I/O port is an abstraction; it corresponds to some memory location that the task will use for sending or receiving data. Data sent or received through a channel is called a non-local data access. A channel is a message queue that connects one task's output port to another task's input port.
    [Show full text]