DOCSLIB.ORG

A High-Performance, Portable Implementation of the MPI Message

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A High-Performance, Portable Implementation of the MPI Message A HighPerformance Portable Implementation of the MPI Message Passing Interface Standard William Gropp Ewing Lusk Mathematics and Computer Science Division Argonne National Lab oratory Nathan Doss Anthony Skjellum Department of Computer Science NSF Engineering Research Center for CFS y Mississippi State University Abstract MPI Message Passing Interface is a sp ecication for a standard library for message passing that was dened by the MPI Forum a broadly based group of parallel com puter vendors library writers and applications sp ecialists Multiple implementations of MPI havebeendevelop ed In this pap er we describ e MPICH unique among existing implementations in its design goal of combining p ortability with high p erformance We do cument its p ortability and p erformance and describ e the architecture bywhichthese features are simultaneously achieved We also discuss the set of to ols that accompany the free distribution of MPICH which constitute the b eginnings of a p ortable parallel programming environment A pro ject of this scop e inevitably imparts lessons ab out parallel computing the sp ecication b eing followed the currenthardware and software environment for parallel computing and pro ject management we describ e those we have learned Finallywe discuss future developments for MPICH including those nec essary to accommo date extensions to the MPI Standard now b eing contemplated by the MPI Forum Intro duction The messagepassing mo del of parallel computation has emerged as an expressive e cient and wellundersto o d paradigm for parallel programming Until recen tly the syntax and precise semantics of each messagepassing library implementation were dierentfrom This work was supp orted by the Mathematical Information and Computational Sciences Division subprogram of the Oce of Computational and Technology Research US Department of Energyunder Contract WEng y This work was supp orted in part by the NSF Engineering ResearchCenter for Computational Field Simulation Mississipp i State University Additional supp ort came from the NSF Career Program under grant ASC and from ARPA under Order D the others although many of the general semantics were similar The proliferation of messagepassing library designs from b oth vendors and users was appropriate for a while but eventually it was seen that enough consensus on requirements and general semantics for messagepassing had b een reached that an attempt at standardization might usefully b e undertaken The pro cess of creating a standard to enable p ortability of messagepassing applica tions co des b egan at a workshop on Message Passing Standardization in April and the Message Passing Interface MPI Forum organized itself at the Sup ercomputing Conference During the next eighteen months the MPI Forum met regularly and Version of the MPI Standard was completed in May Some clarications and re nements were made in the spring of and Version of the MPI Standard is now available For a detailed presentation of the Standard itself see for a tutorial approach to MPI see In this pap er we assume that the reader is relatively familiar with the MPI sp ecication but weprovide a brief overview in Section The pro ject to provide a p ortable implementation of MPI b egan at the same time as the MPI denition pro cess itself The idea was to provide early feedback on decisions b eing made by the MPI Forum and provide an early implementation to allow users to exp eriment with the denitions even as they were b eing develop ed Targets for the implementation were to include all systems capable of supp orting the messagepassing mo del MPICH is a freely available complete implementation of the MPI sp ecication designed to b e b oth p ortable and ecient The CH in MPICH stands for Chameleon symb ol of adaptabilityto ones environment and thus of p ortability Chameleons are fast and from the b eginning a y secondary goal was to give up as little eciency as p ossible for the p ortabilit MPICH is thus b oth a research pro ject and a software development pro ject As a research project its goal is to explore metho ds for narrowing the gap b etween the program mer of a parallel computer and the p erformance deliverable by its hardware In MPICH we adopt the constraint that the programming interface will b e MPI reject constraints on the architecture of the target machine and retain high p erformance measured in terms of bandwidth and latency for messagepassing op erations as a goal As a softwareproject MPICHs goal is to promote the adoption of the MPI Standard by providing users with a free highp erformance implementation on a diversity of platforms while aiding vendors in providing their own customized implementations The extenttowhich these goals have b een achieved is the main thrust of this pap er The rest of this pap er is organized as follows Section gives a short overview of MPI and briey describ es the precursor systems that inuenced MPICH and enabled it to come into existence so quickly In Section we do cument the extentofMPICHsportability and present results of a numb er of p erformance measurements In Section we describ e in some detail the software architecture of MPICH which comprises the results of our researchin combining p ortability and p erformance In Section we presentseveral sp ecic asp ects of the implementation that merit more detailed analysis Section describ es a family of supp orting programs that surround the core MPI implementation and turn MPICH into a p ortable environment for developing parallel applications In Section we describ e howw e as a small distributed group havecombined a numberoffreelyavailable to ols in the Unix environment to enable us to develop distribute and maintain MPICH with a minimum of resources In the course of developing MPICH wehave learned a numb er of lessons from the challenges p osed b oth accidentally and delib erately for MPI implementors by the MPI sp ecication these lessons are discussed in Section Finally Section describ es the current status of MPICH Version as of February and outlines our plans for future development Background In this section we giveanoverview of MPI itself describ e briey the systems on which the rst versions of MPICH were built and review the history of the development of the pro ject Precursor Systems MPICH came into b eing quickly b ecause it could build on stable co de from existing systems These systems pregured in various ways the p ortability p erformance and some of the other features of MPICH Although most of that original co de has b een extensively reworked MPICH still owes some of its design to those earlier systems whichwe briey describ e here P is a thirdgeneration parallel programming library including b oth messagepassing and sharedmemory comp onents p ortable to a great many parallel computing environments including heterogeneous networks Although p contributed much of the co de for TCPIP networks and sharedmemory multipro cessors for the early versions of MPICH most of that has b een rewritten P remains one of the devices on which MPICH can b e built see Section but in most cases more customized alternatives are available Chameleon is a highp erformance p ortabilitypackage for message passing on par allel sup ercomputers It is implemented as a thin layer mostly C macros over vendor messagepassing systems Intels NX TMCs CMMD IBMs MPL for p erformance and over publicly available systems p and PVM for p ortability A substantial amountof to MPICHas detailed in Section Chameleon technology is incorp orated in Zip co de is a p ortable system for writing scalable libraries It contributed several concepts to the design of the MPI Standardin particular contexts groups and mailers the equivalent of MPI communicators Zip co de also contains extensive collective op erations with group scop e as well as virtual top ologies and this co de was heavily b orrowed from in the rst version of MPICH Brief Overview of MPI MPI is a messagepassing application programmer interface together with proto col and semantic sp ecications for how its features must b ehaveinany implementation suchas a message buering and message delivery progress requirement MPI includes p ointto p oint message passing and collective global op erations all scop ed to a usersp ecied group of pro cesses Furthermore MPI provides abstractions for pro cesses at twolevels First pro cesses are named according to the rank of the group in which the communication is b eing p erformed Second virtual top ologies allow for graph or Cartesian naming of pro cesses that help relate the application semantics to the message passing semantics in a convenient ecientway Communicators which house groups and communication context scoping information provide an imp ortant measure of safety that is necessary and useful for building up libraryoriented parallel co de MPI also provides three additional classes of services environmental inquiry basic timing information for application p erformance measurement and a proling interface for external p erformance monitoring MPI makes heterogeneous data conversion a transparent part of its services by requiring datatyp e sp ecication for all communication op erations Both builtin and userdened datatyp es are provided MPI accomplishes its functionality with opaque ob jects with welldened constructors and destructors giving MPI an ob jectbased lo ok and feel Opaque ob jects include groups the fundamental container for pro cesses communicators whichcontain groups and are used as arguments to communication calls
Load more

Recommended publications
  • 2.5 Classification of Parallel Computers
    52 // Architectures 2.5 Classification of Parallel Computers 2.5 Classification of Parallel Computers 2.5.1 Granularity In parallel computing, granularity means the amount of computation in relation to communication or synchronisation Periods of computation are typically separated from periods of communication by synchronization events. • fine level (same operations with different data) ◦ vector processors ◦ instruction level parallelism ◦ fine-grain parallelism: – Relatively small amounts of computational work are done between communication events – Low computation to communication ratio – Facilitates load balancing 53 // Architectures 2.5 Classification of Parallel Computers – Implies high communication overhead and less opportunity for per- formance enhancement – If granularity is too fine it is possible that the overhead required for communications and synchronization between tasks takes longer than the computation. • operation level (different operations simultaneously) • problem level (independent subtasks) ◦ coarse-grain parallelism: – Relatively large amounts of computational work are done between communication/synchronization events – High computation to communication ratio – Implies more opportunity for performance increase – Harder to load balance efficiently 54 // Architectures 2.5 Classification of Parallel Computers 2.5.2 Hardware: Pipelining (was used in supercomputers, e.g. Cray-1) In N elements in pipeline and for 8 element L clock cycles =) for calculation it would take L + N cycles; without pipeline L ∗ N cycles Example of good code for pipelineing: §doi =1 ,k ¤ z ( i ) =x ( i ) +y ( i ) end do ¦ 55 // Architectures 2.5 Classification of Parallel Computers Vector processors, fast vector operations (operations on arrays). Previous example good also for vector processor (vector addition) , but, e.g. recursion – hard to optimise for vector processors Example: IntelMMX – simple vector processor.
    [Show full text]
  • A Massively-Parallel Mixed-Mode Computer Designed to Support
    This paper appeared in th International Parallel Processing Symposium Proc of nd Work shop on Heterogeneous Processing pages NewportBeach CA April Triton A MassivelyParallel MixedMo de Computer Designed to Supp ort High Level Languages Christian G Herter Thomas M Warschko Walter F Tichy and Michael Philippsen University of Karlsruhe Dept of Informatics Postfach D Karlsruhe Germany Mo dula Abstract Mo dula pronounced Mo dulastar is a small ex We present the architectureofTriton a scalable tension of Mo dula for massively parallel program mixedmode SIMDMIMD paral lel computer The ming The programming mo del of Mo dula incor novel features of Triton are p orates b oth data and control parallelism and allows hronous and asynchronous execution mixed sync Support for highlevel machineindependent pro Mo dula is problemorientedinthesensethatthe gramming languages programmer can cho ose the degree of parallelism and mix the control mo de SIMD or MIMDlike as need Fast SIMDMIMD mode switching ed bytheintended algorithm Parallelism maybe nested to arbitrary depth Pro cedures may b e called Special hardware for barrier synchronization of from sequential or parallel contexts and can them multiple process groups selves generate parallel activity without any restric tions Most Mo dula programs can b e translated into ecient co de for b oth SIMD and MIMD archi A selfrouting deadlockfreeperfect shue inter tectures connect with latency hiding Overview of language extensions The architecture is the outcomeofanintegrated de Mo dula extends Mo dula
    [Show full text]
  • Distributed Algorithms with Theoretic Scalability Analysis of Radial and Looped Load flows for Power Distribution Systems
    Electric Power Systems Research 65 (2003) 169Á/177 www.elsevier.com/locate/epsr Distributed algorithms with theoretic scalability analysis of radial and looped load flows for power distribution systems Fangxing Li *, Robert P. Broadwater ECE Department Virginia Tech, Blacksburg, VA 24060, USA Received 15 April 2002; received in revised form 14 December 2002; accepted 16 December 2002 Abstract This paper presents distributed algorithms for both radial and looped load flows for unbalanced, multi-phase power distribution systems. The distributed algorithms are developed from tree-based sequential algorithms. Formulas of scalability for the distributed algorithms are presented. It is shown that computation time dominates communication time in the distributed computing model. This provides benefits to real-time load flow calculations, network reconfigurations, and optimization studies that rely on load flow calculations. Also, test results match the predictions of derived formulas. This shows the formulas can be used to predict the computation time when additional processors are involved. # 2003 Elsevier Science B.V. All rights reserved. Keywords: Distributed computing; Scalability analysis; Radial load flow; Looped load flow; Power distribution systems 1. Introduction Also, the method presented in Ref. [10] was tested in radial distribution systems with no more than 528 buses. Parallel and distributed computing has been applied More recent works [11Á/14] presented distributed to many scientific and engineering computations such as implementations for power flows or power flow based weather forecasting and nuclear simulations [1,2]. It also algorithms like optimizations and contingency analysis. has been applied to power system analysis calculations These works also targeted power transmission systems. [3Á/14].
    [Show full text]
  • Scalability and Performance Management of Internet Applications in the Cloud
    Hasso-Plattner-Institut University of Potsdam Internet Technology and Systems Group Scalability and Performance Management of Internet Applications in the Cloud A thesis submitted for the degree of "Doktors der Ingenieurwissenschaften" (Dr.-Ing.) in IT Systems Engineering Faculty of Mathematics and Natural Sciences University of Potsdam By: Wesam Dawoud Supervisor: Prof. Dr. Christoph Meinel Potsdam, Germany, March 2013 This work is licensed under a Creative Commons License: Attribution – Noncommercial – No Derivative Works 3.0 Germany To view a copy of this license visit http://creativecommons.org/licenses/by-nc-nd/3.0/de/ Published online at the Institutional Repository of the University of Potsdam: URL http://opus.kobv.de/ubp/volltexte/2013/6818/ URN urn:nbn:de:kobv:517-opus-68187 http://nbn-resolving.de/urn:nbn:de:kobv:517-opus-68187 To my lovely parents To my lovely wife Safaa To my lovely kids Shatha and Yazan Acknowledgements At Hasso Plattner Institute (HPI), I had the opportunity to meet many wonderful people. It is my pleasure to thank those who sup- ported me to make this thesis possible. First and foremost, I would like to thank my Ph.D. supervisor, Prof. Dr. Christoph Meinel, for his continues support. In spite of his tight schedule, he always found the time to discuss, guide, and motivate my research ideas. The thanks are extended to Dr. Karin-Irene Eiermann for assisting me even before moving to Germany. I am also grateful for Michaela Schmitz. She managed everything well to make everyones life easier. I owe a thanks to Dr. Nemeth Sharon for helping me to improve my English writing skills.
    [Show full text]
  • Mellanox Scalableshmem Support the Openshmem Parallel Programming Language Over Infiniband
    SOFTWARE PRODUCT BRIEF Mellanox ScalableSHMEM Support the OpenSHMEM Parallel Programming Language over InfiniBand The SHMEM programming library is a one-side communications library that supports a unique set of parallel programming features including point-to-point and collective routines, synchronizations, atomic operations, and a shared memory paradigm used between the processes of a parallel programming application. SHMEM Details API defined by the OpenSHMEM.org consortium. There are two types of models for parallel The library works with the OpenFabrics RDMA programming. The first is the shared memory for Linux stack (OFED), and also has the ability model, in which all processes interact through a to utilize Mellanox Messaging libraries (MXM) globally addressable memory space. The other as well as Mellanox Fabric Collective Accelera- HIGHLIGHTS is a distributed memory model, in which each tions (FCA), providing an unprecedented level of FEATURES processor has its own memory, and interaction scalability for SHMEM programs running over – Use of symmetric variables and one- with another processors memory is done though InfiniBand. sided communication (put/get) message communication. The PGAS model, The use of Mellanox FCA provides for collective – RDMA for performance optimizations or Partitioned Global Address Space, uses a optimizations by taking advantage of the high for one-sided communications combination of these two methods, in which each performance features within the InfiniBand fabric, – Provides shared memory data transfer process has access to its own private memory, including topology aware coalescing, hardware operations (put/get), collective and also to shared variables that make up the operations, and atomic memory multicast and separate quality of service levels for global memory space.
    [Show full text]
  • Scalability and Optimization Strategies for GPU Enhanced Neural Networks (Genn)
    Scalability and Optimization Strategies for GPU Enhanced Neural Networks (GeNN) Naresh Balaji1, Esin Yavuz2, Thomas Nowotny2 {T.Nowotny, E.Yavuz}@sussex.ac.uk, [email protected] 1National Institute of Technology, Tiruchirappalli, India 2School of Engineering and Informatics, University of Sussex, Brighton, UK Abstract: Simulation of spiking neural networks has been traditionally done on high-performance supercomputers or large-scale clusters. Utilizing the parallel nature of neural network computation algorithms, GeNN (GPU Enhanced Neural Network) provides a simulation environment that performs on General Purpose NVIDIA GPUs with a code generation based approach. GeNN allows the users to design and simulate neural networks by specifying the populations of neurons at different stages, their synapse connection densities and the model of individual neurons. In this report we describe work on how to scale synaptic weights based on the configuration of the user-defined network to ensure sufficient spiking and subsequent effective learning. We also discuss optimization strategies particular to GPU computing: sparse representation of synapse connections and occupancy based block-size determination. Keywords: Spiking Neural Network, GPGPU, CUDA, code-generation, occupancy, optimization 1. Introduction The computational performance of traditional single-core CPUs has steadily increased since its arrival, primarily due to the combination of increased clock frequency, process technology advancements and compiler optimizations. The clock frequency was pushed higher and higher, from 5 MHz to 3 GHz in the years from 1983 to 2002, until it reached a stall a few years ago [1] when the power consumption and dissipation of the transistors reached peaks that could not compensated by its cost [2].
    [Show full text]
  • Massively Parallel Computing with CUDA
    Massively Parallel Computing with CUDA Antonino Tumeo Politecnico di Milano 1 GPUs have evolved to the point where many real world applications are easily implemented on them and run significantly faster than on multi-core systems. Future computing architectures will be hybrid systems with parallel-core GPUs working in tandem with multi-core CPUs. Jack Dongarra Professor, University of Tennessee; Author of “Linpack” Why Use the GPU? • The GPU has evolved into a very flexible and powerful processor: • It’s programmable using high-level languages • It supports 32-bit and 64-bit floating point IEEE-754 precision • It offers lots of GFLOPS: • GPU in every PC and workstation What is behind such an Evolution? • The GPU is specialized for compute-intensive, highly parallel computation (exactly what graphics rendering is about) • So, more transistors can be devoted to data processing rather than data caching and flow control ALU ALU Control ALU ALU Cache DRAM DRAM CPU GPU • The fast-growing video game industry exerts strong economic pressure that forces constant innovation GPUs • Each NVIDIA GPU has 240 parallel cores NVIDIA GPU • Within each core 1.4 Billion Transistors • Floating point unit • Logic unit (add, sub, mul, madd) • Move, compare unit • Branch unit • Cores managed by thread manager • Thread manager can spawn and manage 12,000+ threads per core 1 Teraflop of processing power • Zero overhead thread switching Heterogeneous Computing Domains Graphics Massive Data GPU Parallelism (Parallel Computing) Instruction CPU Level (Sequential
    [Show full text]
  • Introduction to Parallel Programming
    Introduction to Parallel Programming Martin Čuma Center for High Performance Computing University of Utah [email protected] Overview • Types of parallel computers. • Parallel programming options. • How to write parallel applications. • How to compile. • How to debug/profile. • Summary, future expansion. 3/13/2009 http://www.chpc.utah.edu Slide 2 Parallel architectures Single processor: • SISD – single instruction single data. Multiple processors: • SIMD - single instruction multiple data. • MIMD – multiple instruction multiple data. Shared Memory Distributed Memory 3/13/2009 http://www.chpc.utah.edu Slide 3 Shared memory Dual quad-core node • All processors have BUS access to local memory CPU Memory • Simpler programming CPU Memory • Concurrent memory access Many-core node (e.g. SGI) • More specialized CPU Memory hardware BUS • CHPC : CPU Memory Linux clusters, 2, 4, 8 CPU Memory core nodes CPU Memory 3/13/2009 http://www.chpc.utah.edu Slide 4 Distributed memory • Process has access only BUS CPU to its local memory Memory • Data between processes CPU Memory must be communicated • More complex Node Network Node programming Node Node • Cheap commodity hardware Node Node • CHPC: Linux clusters Node Node (Arches, Updraft) 8 node cluster (64 cores) 3/13/2009 http://www.chpc.utah.edu Slide 5 Parallel programming options Shared Memory • Threads – POSIX Pthreads, OpenMP – Thread – own execution sequence but shares memory space with the original process • Message passing – processes – Process – entity that executes a program – has its own
    [Show full text]
  • CS650 Computer Architecture Lecture 10 Introduction to Multiprocessors
    NJIT Computer Science Dept CS650 Computer Architecture CS650 Computer Architecture Lecture 10 Introduction to Multiprocessors and PC Clustering Andrew Sohn Computer Science Department New Jersey Institute of Technology Lecture 10: Intro to Multiprocessors/Clustering 1/15 12/7/2003 A. Sohn NJIT Computer Science Dept CS650 Computer Architecture Key Issues Run PhotoShop on 1 PC or N PCs Programmability • How to program a bunch of PCs viewed as a single logical machine. Performance Scalability - Speedup • Run PhotoShop on 1 PC (forget the specs of this PC) • Run PhotoShop on N PCs • Will it run faster on N PCs? Speedup = ? Lecture 10: Intro to Multiprocessors/Clustering 2/15 12/7/2003 A. Sohn NJIT Computer Science Dept CS650 Computer Architecture Types of Multiprocessors Key: Data and Instruction Single Instruction Single Data (SISD) • Intel processors, AMD processors Single Instruction Multiple Data (SIMD) • Array processor • Pentium MMX feature Multiple Instruction Single Data (MISD) • Systolic array • Special purpose machines Multiple Instruction Multiple Data (MIMD) • Typical multiprocessors (Sun, SGI, Cray,...) Single Program Multiple Data (SPMD) • Programming model Lecture 10: Intro to Multiprocessors/Clustering 3/15 12/7/2003 A. Sohn NJIT Computer Science Dept CS650 Computer Architecture Shared-Memory Multiprocessor Processor Prcessor Prcessor Interconnection network Main Memory Storage I/O Lecture 10: Intro to Multiprocessors/Clustering 4/15 12/7/2003 A. Sohn NJIT Computer Science Dept CS650 Computer Architecture Distributed-Memory Multiprocessor Processor Processor Processor IO/S MM IO/S MM IO/S MM Interconnection network IO/S MM IO/S MM IO/S MM Processor Processor Processor Lecture 10: Intro to Multiprocessors/Clustering 5/15 12/7/2003 A.
    [Show full text]
  • CS 677: Parallel Programming for Many-Core Processors Lecture 1
    1 CS 677: Parallel Programming for Many-core Processors Lecture 1 Instructor: Philippos Mordohai Webpage: mordohai.github.io E-mail: [email protected] Objectives • Learn how to program massively parallel processors and achieve – High performance – Functionality and maintainability – Scalability across future generations • Acquire technical knowledge required to achieve the above goals – Principles and patterns of parallel programming – Processor architecture features and constraints – Programming API, tools and techniques 2 Important Points • This is an elective course. You chose to be here. • Expect to work and to be challenged. • If your programming background is weak, you will probably suffer. • This course will evolve to follow the rapid pace of progress in GPU programming. It is bound to always be a little behind… 3 Important Points II • At any point ask me WHY? • You can ask me anything about the course in class, during a break, in my office, by email. – If you think a homework is taking too long or is wrong. – If you can’t decide on a project. 4 Logistics • Class webpage: http://mordohai.github.io/classes/cs677_s20.html • Office hours: Tuesdays 5-6pm and by email • Evaluation: – Homework assignments (40%) – Quizzes (10%) – Midterm (15%) – Final project (35%) 5 Project • Pick topic BEFORE middle of the semester • I will suggest ideas and datasets, if you can’t decide • Deliverables: – Project proposal – Presentation in class – Poster in CS department event – Final report (around 8 pages) 6 Project Examples • k-means • Perceptron • Boosting – General – Face detector (group of 2) • Mean Shift • Normal estimation for 3D point clouds 7 More Ideas • Look for parallelizable problems in: – Image processing – Cryptanalysis – Graphics • GPU Gems – Nearest neighbor search 8 Even More… • Particle simulations • Financial analysis • MCMC • Games/puzzles 9 Resources • Textbook – Kirk & Hwu.
    [Show full text]
  • An Open Standard for SHMEM Implementations Introduction
    OpenSHMEM: An Open Standard for SHMEM Implementations Introduction • OpenSHMEM is an effort to create a standardized SHMEM library for C , C++, and Fortran • OpenSHMEM is a Partitioned Global Address Space (PGAS) library and supports Single Program Multiple Data (SPMD) style of programming • SGI’s SHMEM API is the baseline for OpenSHMEM Specification 1.0 • One-sided communication is achieved through Symmetric variables • globals C/C++: Non-stack variables Fortran: objects in common blocks or with the SAVE attribute • dynamically allocated and managed by the OpenSHMEM Library Figure 1. Dynamic allocation of symmetric variable ‘x’ OpenSHMEM API • OpenSHMEM Specification 1.0 provides support for data transfer operations, collective and point- to-point synchronization mechanisms (barrier, fence, quiet, wait), collective operations (broadcast, collection, reduction), atomic memory operations (swap, fetch/add, increment), and distributed locks. • Open to the community for reviews and contributions. Current Status • University of Houston has developed a portable reference OpenSHMEM library. • OpenSHMEM Specification 1.0 man pages are available. • OpenSHMEM mailing list for discussions and contributions can be joined by emailing [email protected] • Website for OpenSHMEM and a Wiki for community use are available. Figure 2. University of Houston http://www.openshmem.org/ Implementation Future Work and Goals • Develop OpenSHMEM Specification 2.0 with co-operation and contribution from the OpenSHMEM community • Discuss and extend specification with important new capabilities • Improve on the OpenSHMEM reference implementation by providing support for scalable collective algorithms • Explore innovative hardware platforms .
    [Show full text]
  • What Is SPMD? Messages
    1 2 Outline Motivation for MPI Overview of PVM and MPI The pro cess that pro duced MPI What is di erent ab out MPI? { the \usual" send/receive Jack Dongarra { the MPI send/receive { simple collective op erations Computer Science Department New in MPI: Not in MPI UniversityofTennessee Some simple complete examples, in Fortran and C and Communication mo des, more on collective op erations Implementation status Mathematical Sciences Section Oak Ridge National Lab oratory MPICH - a free, p ortable implementation MPI resources on the Net MPI-2 http://www.netlib.org/utk/p eople/JackDongarra.html 3 4 Messages What is SPMD? 2 Messages are packets of data moving b etween sub-programs. 2 Single Program, Multiple Data 2 The message passing system has to b e told the 2 Same program runs everywhere. following information: 2 Restriction on the general message-passing mo del. { Sending pro cessor 2 Some vendors only supp ort SPMD parallel programs. { Source lo cation { Data typ e 2 General message-passing mo del can b e emulated. { Data length { Receiving pro cessors { Destination lo cation { Destination size 5 6 Access Point-to-Point Communication 2 A sub-program needs to b e connected to a message passing 2 Simplest form of message passing. system. 2 One pro cess sends a message to another 2 A message passing system is similar to: 2 Di erenttyp es of p oint-to p oint communication { Mail b ox { Phone line { fax machine { etc. 7 8 Synchronous Sends Asynchronous Sends Provide information about the completion of the Only know when the message has left.
    [Show full text]