HPVM: Heterogeneous Parallel Virtual Machine

Total Page:16

File Type:pdf, Size:1020Kb

HPVM: Heterogeneous Parallel Virtual Machine HPVM: Heterogeneous Parallel Virtual Machine Maria Kotsifakou∗ Prakalp Srivastava∗ Matthew D. Sinclair Department of Computer Science Department of Computer Science Department of Computer Science University of Illinois at University of Illinois at University of Illinois at Urbana-Champaign Urbana-Champaign Urbana-Champaign [email protected] [email protected] [email protected] Rakesh Komuravelli Vikram Adve Sarita Adve Qualcomm Technologies Inc. Department of Computer Science Department of Computer Science [email protected]. University of Illinois at University of Illinois at com Urbana-Champaign Urbana-Champaign [email protected] [email protected] Abstract hardware, and that runtime scheduling policies can make We propose a parallel program representation for heteroge- use of both program and runtime information to exploit the neous systems, designed to enable performance portability flexible compilation capabilities. Overall, we conclude that across a wide range of popular parallel hardware, including the HPVM representation is a promising basis for achieving GPUs, vector instruction sets, multicore CPUs and poten- performance portability and for implementing parallelizing tially FPGAs. Our representation, which we call HPVM, is a compilers for heterogeneous parallel systems. hierarchical dataflow graph with shared memory and vector CCS Concepts • Computer systems organization → instructions. HPVM supports three important capabilities for Heterogeneous (hybrid) systems; programming heterogeneous systems: a compiler interme- diate representation (IR), a virtual instruction set (ISA), and Keywords Virtual ISA, Compiler, Parallel IR, Heterogeneous a basis for runtime scheduling; previous systems focus on Systems, GPU, Vector SIMD only one of these capabilities. As a compiler IR, HPVM aims to enable effective code generation and optimization for het- 1 Introduction erogeneous systems. As a virtual ISA, it can be used to ship Heterogeneous parallel systems are becoming increasingly executable programs, in order to achieve both functional popular in systems ranging from portable mobile devices to portability and performance portability across such systems. high-end supercomputers to data centers. Such systems are At runtime, HPVM enables flexible scheduling policies, both attractive because they use specialized computing elements, through the graph structure and the ability to compile indi- including GPUs, vector hardware, FPGAs, and domain- vidual nodes in a program to any of the target devices on a specific accelerators, that can greatly improve energy ef- system. We have implemented a prototype HPVM system, ficiency, performance, or both, compared with traditional defining the HPVM IR as an extension of the LLVM compiler homogeneous systems. A major drawback, however, is that IR, compiler optimizations that operate directly on HPVM programming heterogeneous systems is extremely challeng- graphs, and code generators that translate the virtual ISA to ing at multiple levels: algorithm designers, application devel- NVIDIA GPUs, Intel’s AVX vector units, and to multicore opers, parallel language designers, compiler developers and X86-64 processors. Experimental results show that HPVM op- hardware engineers must all reason about performance, scal- timizations achieve significant performance improvements, ability, and portability across many different combinations HPVM translators achieve performance competitive with of diverse parallel hardware. manually developed OpenCL code for both GPUs and vector At a fundamental level, we believe these challenges arise from three root causes: (1) diverse hardware parallelism mod- ∗The first two authors contributed equally to the paper. els; (2) diverse memory architectures; and (3) diverse hard- Permission to make digital or hard copies of part or all of this work for ware instruction sets. Some widely used heterogeneous sys- personal or classroom use is granted without fee provided that copies are tems, such as GPUs, partially address these problems by not made or distributed for profit or commercial advantage and that copies defining a virtual instruction set (ISA) spanning one or more bear this notice and the full citation on the first page. Copyrights for third- families of devices, e.g., PTX for NVIDIA GPUs, HSAIL for party components of this work must be honored. For all other uses, contact GPUs from several vendors and SPIR for devices running the owner/author(s). OpenCL. Software can be shipped in virtual ISA form and PPoPP ’18, February 24–28, 2018, Vienna, Austria © 2018 Copyright held by the owner/author(s). then translated to the native ISA for execution on a supported ACM ISBN 978-1-4503-4982-6/18/02. device within the target family at install time or runtime, https://doi.org/10.1145/3178487.3178493 thus achieving portability of “virtual object code” across the PPoPP ’18, February 24–28, 2018, Vienna, Austria Kotsifakou, Srivastava, Sinclair, Komuravelli, V. Adve, and S. Adve corresponding family of devices. Except for SPIR, which is The parallel program representation we propose is a hi- essentially a lower-level representation of the OpenCL lan- erarchical dataflow graph with shared memory. The graph guage, these virtual ISAs are primarily focused on GPUs nodes can represent either coarse-grain or fine-grain compu- and do not specifically address other hardware classes, like tational tasks, although we focus on moderately coarse-grain vector hardware or FPGAs. Moreover, none of these virtual tasks (such as an entire inner-loop iteration) in this work. The ISAs aim to address the other challenges, such as algorithm dataflow graph edges capture explicit data transfers between design, language design, and compiler optimizations, across nodes, while ordinary load and store instructions express diverse heterogeneous devices. implicit communication via shared memory. The graph is We believe that these challenges can be best addressed by hierarchical because a node may contain another dataflow developing a single parallel program representation flexible graph. The leaf nodes can contain both scalar and vector enough to support at least three different purposes: (1) A com- computations. A graph node represents a static computation, piler intermediate representation, for compiler optimizations and any such node can be “instantiated” in a rectangular grid and code generation for diverse heterogeneous hardware. of dynamic node instances, representing independent parallel Such a compiler IR must be able to implement a wide range instances of the computation (in which case, the incident of different parallel languages, including general-purpose edges are instantiated as well, as described later). ones like OpenMP, CUDA and OpenCL, and domain-specific The hierarchical dataflow graphs naturally capture all the ones like Halide and TensorFlow. (2) A virtual ISA, to allow important kinds of coarse- and fine-grain data and task par- virtual object code to be shipped and then translated down allelism in heterogeneous systems. In particular, the graph to native code for different heterogeneous system configu- structure captures coarse-grain task parallelism (including rations. This requirement is essential to enable application pipelined parallelism in streaming computations); the graph teams to develop and ship application code for multiple de- hierarchy captures multiple levels and granularities of nested vices within a family. (3) A representation for runtime sched- parallelism; the node instantiation mechanism captures ei- uling, to enable flexible mapping and load-balancing policies, ther coarse- or fine-grain SPMD-style data parallelism; and in order to accommodate static variations among different explicit vector instructions within leaf nodes capture fine- compute kernels and dynamic variations due to external ef- grain vector parallelism (this can also be generated by auto- fects like energy fluctuations or job arrivals and departures. matic vectorization across independent node instances). We believe that a representation that can support all these We describe a prototype system (also called HPVM) that three capabilities could (in future) also simplify parallel algo- supports all three capabilities listed earlier. The system de- rithm development and influence parallel language design, fines a compiler IR as an extension of the LLVM IR[30] by although we do not explore those in this work. adding HPVM abstractions as a higher-level layer describing In this paper, we propose such a parallel program repre- the parallel structure of a program. sentation, Heterogeneous Parallel Virtual Machine (HPVM), As examples of the use of HPVM as a compiler IR, we and evaluate it for three classes of parallel hardware: GPUs, have implemented two illustrative compiler optimizations, SIMD vector instructions, and multicore CPUs. (In ongoing graph node fusion and tiling, both of which operate directly work, we are also targeting FPGAs using the same program on the HPVM dataflow graphs. Node fusion achieves “kernel representation.) Our evaluation shows that HPVM can serve fusion”, and the graph structure makes it explicit when it all three purposes listed above: a compiler IR, a virtual ISA, is safe to fuse two or more nodes. Similarly (and somewhat and a scheduling representation, as described below. surprisingly), we find that the graph hierarchy is alsoan No previous system we know of can achieve all three pur- effective and portable method to capture tiling of computa- poses,
Recommended publications
  • A Case for High Performance Computing with Virtual Machines
    A Case for High Performance Computing with Virtual Machines Wei Huangy Jiuxing Liuz Bulent Abaliz Dhabaleswar K. Panday y Computer Science and Engineering z IBM T. J. Watson Research Center The Ohio State University 19 Skyline Drive Columbus, OH 43210 Hawthorne, NY 10532 fhuanwei, [email protected] fjl, [email protected] ABSTRACT in the 1960s [9], but are experiencing a resurgence in both Virtual machine (VM) technologies are experiencing a resur- industry and research communities. A VM environment pro- gence in both industry and research communities. VMs of- vides virtualized hardware interfaces to VMs through a Vir- fer many desirable features such as security, ease of man- tual Machine Monitor (VMM) (also called hypervisor). VM agement, OS customization, performance isolation, check- technologies allow running different guest VMs in a phys- pointing, and migration, which can be very beneficial to ical box, with each guest VM possibly running a different the performance and the manageability of high performance guest operating system. They can also provide secure and computing (HPC) applications. However, very few HPC ap- portable environments to meet the demanding requirements plications are currently running in a virtualized environment of computing resources in modern computing systems. due to the performance overhead of virtualization. Further, Recently, network interconnects such as InfiniBand [16], using VMs for HPC also introduces additional challenges Myrinet [24] and Quadrics [31] are emerging, which provide such as management and distribution of OS images. very low latency (less than 5 µs) and very high bandwidth In this paper we present a case for HPC with virtual ma- (multiple Gbps).
    [Show full text]
  • Introduction to Openacc 2018 HPC Workshop: Parallel Programming
    Introduction to OpenACC 2018 HPC Workshop: Parallel Programming Alexander B. Pacheco Research Computing July 17 - 18, 2018 CPU vs GPU CPU : consists of a few cores optimized for sequential serial processing GPU : has a massively parallel architecture consisting of thousands of smaller, more efficient cores designed for handling multiple tasks simultaneously GPU enabled applications 2 / 45 CPU vs GPU CPU : consists of a few cores optimized for sequential serial processing GPU : has a massively parallel architecture consisting of thousands of smaller, more efficient cores designed for handling multiple tasks simultaneously GPU enabled applications 2 / 45 CPU vs GPU CPU : consists of a few cores optimized for sequential serial processing GPU : has a massively parallel architecture consisting of thousands of smaller, more efficient cores designed for handling multiple tasks simultaneously GPU enabled applications 2 / 45 CPU vs GPU CPU : consists of a few cores optimized for sequential serial processing GPU : has a massively parallel architecture consisting of thousands of smaller, more efficient cores designed for handling multiple tasks simultaneously GPU enabled applications 2 / 45 3 / 45 Accelerate Application for GPU 4 / 45 GPU Accelerated Libraries 5 / 45 GPU Programming Languages 6 / 45 What is OpenACC? I OpenACC Application Program Interface describes a collection of compiler directive to specify loops and regions of code in standard C, C++ and Fortran to be offloaded from a host CPU to an attached accelerator. I provides portability across operating systems, host CPUs and accelerators History I OpenACC was developed by The Portland Group (PGI), Cray, CAPS and NVIDIA. I PGI, Cray, and CAPs have spent over 2 years developing and shipping commercial compilers that use directives to enable GPU acceleration as core technology.
    [Show full text]
  • GPU Computing with Openacc Directives
    Introduction to OpenACC Directives Duncan Poole, NVIDIA Thomas Bradley, NVIDIA GPUs Reaching Broader Set of Developers 1,000,000’s CAE CFD Finance Rendering Universities Data Analytics Supercomputing Centers Life Sciences 100,000’s Oil & Gas Defense Weather Research Climate Early Adopters Plasma Physics 2004 Present Time 3 Ways to Accelerate Applications Applications OpenACC Programming Libraries Directives Languages “Drop-in” Easily Accelerate Maximum Acceleration Applications Flexibility 3 Ways to Accelerate Applications Applications OpenACC Programming Libraries Directives Languages CUDA Libraries are interoperable with OpenACC “Drop-in” Easily Accelerate Maximum Acceleration Applications Flexibility 3 Ways to Accelerate Applications Applications OpenACC Programming Libraries Directives Languages CUDA Languages are interoperable with OpenACC, “Drop-in” Easily Accelerate too! Maximum Acceleration Applications Flexibility NVIDIA cuBLAS NVIDIA cuRAND NVIDIA cuSPARSE NVIDIA NPP Vector Signal GPU Accelerated Matrix Algebra on Image Processing Linear Algebra GPU and Multicore NVIDIA cuFFT Building-block Sparse Linear C++ STL Features IMSL Library Algorithms for CUDA Algebra for CUDA GPU Accelerated Libraries “Drop-in” Acceleration for Your Applications OpenACC Directives CPU GPU Simple Compiler hints Program myscience Compiler Parallelizes code ... serial code ... !$acc kernels do k = 1,n1 do i = 1,n2 OpenACC ... parallel code ... Compiler Works on many-core GPUs & enddo enddo Hint !$acc end kernels multicore CPUs ... End Program myscience
    [Show full text]
  • Openacc Course October 2017. Lecture 1 Q&As
    OpenACC Course October 2017. Lecture 1 Q&As. Question Response I am currently working on accelerating The GCC compilers are lagging behind the PGI compilers in terms of code compiled in gcc, in your OpenACC feature implementations so I'd recommend that you use the PGI experience should I grab the gcc-7 or compilers. PGI compilers provide community version which is free and you try to compile the code with the pgi-c ? can use it to compile OpenACC codes. New to OpenACC. Other than the PGI You can find all the supported compilers here, compiler, what other compilers support https://www.openacc.org/tools OpenACC? Is it supporting Nvidia GPU on TK1? I Currently there isn't an OpenACC implementation that supports ARM think, it must be processors including TK1. PGI is considering adding this support sometime in the future, but for now, you'll need to use CUDA. OpenACC directives work with intel Xeon Phi is treated as a MultiCore x86 CPU. With PGI you can use the "- xeon phi platform? ta=multicore" flag to target multicore CPUs when using OpenACC. Does it have any application in field of In my opinion OpenACC enables good software engineering practices by Software Engineering? enabling you to write a single source code, which is more maintainable than having to maintain different code bases for CPUs, GPUs, whatever. Does the CPU comparisons include I think that the CPU comparisons include standard vectorisation that the simd vectorizations? PGI compiler applies, but no specific hand-coded vectorisation optimisations or intrinsic work. Does OpenMP also enable OpenMP does have support for GPUs, but the compilers are just now parallelization on GPU? becoming available.
    [Show full text]
  • Exploring Massive Parallel Computation with GPU
    Need for parallelism Graphical Processor Units Gravitational Microlensing Modelling Exploring Massive Parallel Computation with GPU Ian Bond Massey University, Auckland, New Zealand 2011 Sagan Exoplanet Workshop Pasadena, July 25-29 2011 Ian Bond | Microlensing parallelism 1/40 Need for parallelism Graphical Processor Units Gravitational Microlensing Modelling Assumptions/Purpose You are all involved in microlensing modelling and you have (or are working on) your own code this lecture shows how to get started on getting code to run on a GPU then its over to you . Ian Bond | Microlensing parallelism 2/40 Need for parallelism Graphical Processor Units Gravitational Microlensing Modelling Outline 1 Need for parallelism 2 Graphical Processor Units 3 Gravitational Microlensing Modelling Ian Bond | Microlensing parallelism 3/40 Need for parallelism Graphical Processor Units Gravitational Microlensing Modelling Paralel Computing Parallel Computing is use of multiple computers, or computers with multiple internal processors, to solve a problem at a greater computational speed than using a single computer (Wilkinson 2002). How does one achieve parallelism? Ian Bond | Microlensing parallelism 4/40 Need for parallelism Graphical Processor Units Gravitational Microlensing Modelling Grand Challenge Problems A grand challenge problem is one that cannot be solved in a reasonable amount of time with todays computers’ Examples: – Modelling large DNA structures – Global weather forecasting – N body problem (N very large) – brain simulation Has microlensing
    [Show full text]
  • Dcuda: Hardware Supported Overlap of Computation and Communication
    dCUDA: Hardware Supported Overlap of Computation and Communication Tobias Gysi Jeremia Bar¨ Torsten Hoefler Department of Computer Science Department of Computer Science Department of Computer Science ETH Zurich ETH Zurich ETH Zurich [email protected] [email protected] [email protected] Abstract—Over the last decade, CUDA and the underlying utilization of the costly compute and network hardware. To GPU hardware architecture have continuously gained popularity mitigate this problem, application developers can implement in various high-performance computing application domains manual overlap of computation and communication [23], [27]. such as climate modeling, computational chemistry, or machine learning. Despite this popularity, we lack a single coherent In particular, there exist various approaches [13], [22] to programming model for GPU clusters. We therefore introduce overlap the communication with the computation on an inner the dCUDA programming model, which implements device- domain that has no inter-node data dependencies. However, side remote memory access with target notification. To hide these code transformations significantly increase code com- instruction pipeline latencies, CUDA programs over-decompose plexity which results in reduced real-world applicability. the problem and over-subscribe the device by running many more threads than there are hardware execution units. Whenever a High-performance system design often involves trading thread stalls, the hardware scheduler immediately proceeds with off sequential performance against parallel throughput. The the execution of another thread ready for execution. This latency architectural difference between host and device processors hiding technique is key to make best use of the available hardware perfectly showcases the two extremes of this design space.
    [Show full text]
  • On the Virtualization of CUDA Based GPU Remoting on ARM and X86 Machines in the Gvirtus Framework
    On the Virtualization of CUDA Based GPU Remoting on ARM and X86 Machines in the GVirtuS Framework Montella, R., Giunta, G., Laccetti, G., Lapegna, M., Palmieri, C., Ferraro, C., Pelliccia, V., Hong, C-H., Spence, I., & Nikolopoulos, D. (2017). On the Virtualization of CUDA Based GPU Remoting on ARM and X86 Machines in the GVirtuS Framework. International Journal of Parallel Programming, 45(5), 1142-1163. https://doi.org/10.1007/s10766-016-0462-1 Published in: International Journal of Parallel Programming Document Version: Peer reviewed version Queen's University Belfast - Research Portal: Link to publication record in Queen's University Belfast Research Portal Publisher rights © 2016 Springer Verlag. The final publication is available at Springer via http://dx.doi.org/ 10.1007/s10766-016-0462-1 General rights Copyright for the publications made accessible via the Queen's University Belfast Research Portal is retained by the author(s) and / or other copyright owners and it is a condition of accessing these publications that users recognise and abide by the legal requirements associated with these rights. Take down policy The Research Portal is Queen's institutional repository that provides access to Queen's research output. Every effort has been made to ensure that content in the Research Portal does not infringe any person's rights, or applicable UK laws. If you discover content in the Research Portal that you believe breaches copyright or violates any law, please contact [email protected]. Download date:08. Oct. 2021 Noname manuscript No. (will be inserted by the editor) On the virtualization of CUDA based GPU remoting on ARM and X86 machines in the GVirtuS framework Raffaele Montella · Giulio Giunta · Giuliano Laccetti · Marco Lapegna · Carlo Palmieri · Carmine Ferraro · Valentina Pelliccia · Cheol-Ho Hong · Ivor Spence · Dimitrios S.
    [Show full text]
  • Multi-Threaded GPU Accelerration of ORBIT with Minimal Code
    Princeton Plasma Physics Laboratory PPPL- 4996 4996 Multi-threaded GPU Acceleration of ORBIT with Minimal Code Modifications Ante Qu, Stephane Ethier, Eliot Feibush and Roscoe White FEBRUARY, 2014 Prepared for the U.S. Department of Energy under Contract DE-AC02-09CH11466. Princeton Plasma Physics Laboratory Report Disclaimers Full Legal Disclaimer This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, nor any of their contractors, subcontractors or their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or any third party’s use or the results of such use of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof or its contractors or subcontractors. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof. Trademark Disclaimer Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof or its contractors or subcontractors. PPPL Report Availability Princeton Plasma Physics Laboratory: http://www.pppl.gov/techreports.cfm Office of Scientific and Technical Information (OSTI): http://www.osti.gov/bridge Related Links: U.S.
    [Show full text]
  • Openacc Getting Started Guide
    OPENACC GETTING STARTED GUIDE Version 2018 TABLE OF CONTENTS Chapter 1. Overview............................................................................................ 1 1.1. Terms and Definitions....................................................................................1 1.2. System Prerequisites..................................................................................... 2 1.3. Prepare Your System..................................................................................... 2 1.4. Supporting Documentation and Examples............................................................ 3 Chapter 2. Using OpenACC with the PGI Compilers...................................................... 4 2.1. OpenACC Directive Summary........................................................................... 4 2.2. CUDA Toolkit Versions....................................................................................6 2.3. C Structs in OpenACC....................................................................................8 2.4. C++ Classes in OpenACC.................................................................................9 2.5. Fortran Derived Types in OpenACC...................................................................13 2.6. Fortran I/O............................................................................................... 15 2.6.1. OpenACC PRINT Example......................................................................... 15 2.7. OpenACC Atomic Support.............................................................................
    [Show full text]
  • Productive High Performance Parallel Programming with Auto-Tuned Domain-Specific Embedded Languages
    Productive High Performance Parallel Programming with Auto-tuned Domain-Specific Embedded Languages By Shoaib Ashraf Kamil A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate Division of the University of California, Berkeley Committee in charge: Professor Armando Fox, Co-Chair Professor Katherine Yelick, Co-Chair Professor James Demmel Professor Berend Smit Fall 2012 Productive High Performance Parallel Programming with Auto-tuned Domain-Specific Embedded Languages Copyright c 2012 Shoaib Kamil. Abstract Productive High Performance Parallel Programming with Auto-tuned Domain-Specific Embedded Languages by Shoaib Ashraf Kamil Doctor of Philosophy in Computer Science University of California, Berkeley Professor Armando Fox, Co-Chair Professor Katherine Yelick, Co-Chair As the complexity of machines and architectures has increased, performance tuning has become more challenging, leading to the failure of general compilers to generate the best possible optimized code. Expert performance programmers can often hand-write code that outperforms compiler- optimized low-level code by an order of magnitude. At the same time, the complexity of programs has also increased, with modern programs built on a variety of abstraction layers to manage complexity, yet these layers hinder efforts at optimization. In fact, it is common to lose one or two additional orders of magnitude in performance when going from a low-level language such as Fortran or C to a high-level language like Python, Ruby, or Matlab. General purpose compilers are limited by the inability of program analysis to determine pro- grammer intent, as well as the lack of detailed performance models that always determine the best executable code for a given computation and architecture.
    [Show full text]
  • Introduction to GPU Programming with CUDA and Openacc
    Introduction to GPU Programming with CUDA and OpenACC Alabama Supercomputer Center 1 Alabama Research and Education Network Contents Topics § Why GPU chips and CUDA? § GPU chip architecture overview § CUDA programming § Queue system commands § Other GPU programming options § OpenACC programming § Comparing GPUs to other processors 2 What is a GPU chip? GPU § A Graphic Processing Unit (GPU) chips is an adaptation of the technology in a video rendering chip to be used as a math coprocessor. § The earliest graphic cards simply mapped memory bytes to screen pixels – i.e. the Apple ][ in 1980. § The next generation of graphics cards (1990s) had 2D rendering capabilities for rendering lines and shaded areas. § Graphics cards started accelerating 3D rendering with standards like OpenGL and DirectX in the early 2000s. § The most recent graphics cards have programmable processors, so that game physics can be offloaded from the main processor to the GPU. § A series of GPU chips sometimes called GPGPU (General Purpose GPU) have double precision capability so that they can be used as math coprocessors. 3 Why GPUs? GPU Comparison of peak theoretical GFLOPs and memory bandwidth for NVIDIA GPUs and Intel CPUs over the past few years. Graphs from the NVIDIA CUDA C Programming Guide 4.0. 4 CUDA Programming Language CUDA The GPU chips are massive multithreaded, manycore SIMD processors. SIMD stands for Single Instruction Multiple Data. Previously chips were programmed using standard graphics APIs (DirectX, OpenGL). CUDA, an extension of C, is the most popular GPU programming language. CUDA can also be called from a C++ program. The CUDA standard has no FORTRAN support, but Portland Group sells a third party CUDA FORTRAN.
    [Show full text]
  • Parallel Processing Using PVM on a Linux Cluster
    ParallelParallel ProcessingProcessing usingusing PVMPVM onon aa LinuxLinux ClusterCluster ThomasThomas K.K. GederbergGederberg CENGCENG 65326532 FallFall 20072007 WhatWhat isis PVM?PVM? PVM (Parallel Virtual Machine) is a “software system that permits a heterogeneous collection of Unix computers networked together to be viewed by a user’s program as a single parallel computer”.1 Since the distributed computers have no shared memory, the PVM system uses Message Passing for data communication among the processors. PVM and MPI (Message Passing Interface) are the two common message passing API’s for distributed computing. MPI, the newer of the two API’s, was designed for homogenous distributed computers and offers higher performance. PVM, however, sacrifices performance for flexibility. PVM was designed to run over a heterogeneous network of distributed computers. PVM also allows for fault tolerance – it allows for the detection of, and reconfiguration for, node failures. PVM includes both C and Fortran libraries. PVM was developed by Oak Ridge National Laboratory, the University of Tennessee, Emory University, and Carnegie Mellon University. 2 InstallingInstalling PVMPVM inin LinuxLinux PVM is easy to install on a machine running Linux and can be installed in the users home directory (does not need root or administrator privileges). Obtain the PVM source code (latest version is 3.4.5) from the PVM website: http://www.csm.ornl.gov/pvm/pvm_home.html Unzip and untar the package (pvm3.4.5.tgz) in your home directory – this will create a directory called pvm3 Modify your startup script (.bashrc, .cshrc, etc.) to define: $PVM_ROOT = $HOME/pvm3 $PVM_ARCH = LINUX cd to the pvm3 directory Type make The makefile will build pvm (the PVM console), pvmd3 (the pvm daemon), libpvm3.a (PVM C/C++ library), libfpvm3.a (PVM Fortran library), and libgpvm3.a (PVM group library) and places all of these files in the $PVM_ROOT/lib/LINUX directory.
    [Show full text]