DOCSLIB.ORG

Redalyc.Parallel Programming Languages on Heterogeneous

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Redalyc.Parallel Programming Languages on Heterogeneous Tecnura ISSN: 0123-921X [email protected] Universidad Distrital Francisco José de Caldas Colombia Hernández B, Esteban; Montoya Gaviria, Gerardo de Jesús; Montenegro, Carlos Enrique Parallel programming languages on heterogeneous architectures using openmpc, ompss, openacc and openmp Tecnura, vol. 18, núm. 2, diciembre, 2014, pp. 160-170 Universidad Distrital Francisco José de Caldas Bogotá, Colombia Available in: http://www.redalyc.org/articulo.oa?id=257059813014 How to cite Complete issue Scientific Information System More information about this article Network of Scientific Journals from Latin America, the Caribbean, Spain and Portugal Journal's homepage in redalyc.org Non-profit academic project, developed under the open access initiative Tecnura http://revistas.udistrital.edu.co/ojs/index.php/Tecnura/issue/view/687 DOI: http://doi.org/10.14483/udistrital.jour.tecnura.2014.DSE1.a14 REFLEXIÓN Parallel programming languages on heterogeneous architectures using openmpc, ompss, openacc and openmp Lenguajes para programación paralela en arquitecturas heterogéneas utilizando openmpc, ompss, openacc y openmp Esteban Hernández B.*, Gerardo de Jesús Montoya Gaviria**, Carlos Enrique Montenegro*** Fecha de recepción: June 10tyh, 2014 Fecha de aceptación: November 4t, 2014 Citation / Para citar este artículo: Hernández, E., Gaviria, G. de J. M., & Montenegro, C. E. (2014). Para- llel programming languages on heterogeneous architectures using OPENMPC, OMPSS, OPENACC and OPENMP. Revista Tecnura, 18 (Edición especial doctorado), 160–170. doi: 10.14483/udistrital.jour.tec- nura.2014.DSE1.a14 RESUMEN ABSTRACT En el campo de la programación paralela, ha arri- On the field of parallel programing has emerged a bado un nuevo gran jugador en los últimos 10 new big player in the last 10 years. The GPU’s have años. Las GPU han tomado una importancia re- taken a relevant importance on scientific computing levante en la computación científica debido a because they offer a high performance computing, que ofrecen alto rendimiento computacional, low cost and simplicity of implementation. Howe- bajo costos y simplicidad de implementación; sin ver, one of the most important challenges is the pro- embargo, uno de los desafíos más grandes que gram languages used for this devices. The effort for poseen son los lenguajes utilizados para la progra- recoding algorithms designed for CPUs is a critical mación de los dispositivos. El esfuerzo de reescri- problem. In this paper we review three of principal bir algoritmos diseñados originalmente para CPU frameworks for programming CUDA devices com- es uno de los mayores problemas. En este artículo pared with the new directives introduced on the se revisan tres frameworks de programación para OpenMP 4 standard resolving the Jacobi iterative la tecnología CUDA y se realiza una compara- method. ción con el reciente estándar OpenMP versión 4, Keywords: CUDA, Jacobbi method, OmpSS, Ope- resolviendo el método iterativo de Jacobi. nACC, OpenMP, OpenMP, Parallel Programming. Palabras clave: Método de Jacobi, OmpSS, Ope- nACC, OpenMP, Programación paralela. * Network Engineering with Master degree on software engineering and Free Software construction, minor degree on applied mathematics and network software construction. Now running Doctorate studies on Engineering at Universidad Distrital and works as principal architect on RUNT. Now works in focus on Parallel Programming, high performance computing, computational numerical simulation and numerical weather forecast. E-mail: [email protected] ** Engineer in meteorology from the University of Leningrad, with a doctorate in physical-mathematical sciences of State Moscow University, pioneer in the area of meteorology in Colombia, in charge of meteorology graduate at Universidad Nacional de Colombia, researcher and director of more than 12 graduate theses in meteorology dynamic area and numerical forecast, air quality, efficient use of climate models and weather. He is currently a full professor of the Faculty of Geosciences at Universidad Nacional de Colombia. E-mail: gdmontoyag@ unal.edu.co *** System Engineering, PhD and Master degree on Informatics, director of research group GIIRA with focus on Social Network Analyzing, eLearning and data visualization. He is currently associate professor of Engineering Faculty at Universidad Distrital. E-mail: cemonte- [email protected] Tecnura • p-ISSN: 0123-921X • e-ISSN: 2248-7638 • Vol. 18 - Special Edition Doctorate • December 2014 • pp. 160-170 [ 160 ] Parallel programming languages on heterogeneous architectures using openmpc, ompss, openacc and openmp Esteban Hernández B., Gerardo de Jesús Montoya Gaviria, Carlos Enrique Montenegro INTRODUCTION is framework focusses on task decomposition para- digm for developing parallel applications on cluster Since 10 years ago, the massively parallel proces- environments with heterogeneous architectures. It sors have used the GPUs as principal element on provides a set of compiler directives that can be the new approach in parallel programming; it’s used to annotate a sequential code. Additional fea- evolved from a graphics-specific accelerator to a tures have been added to support the use of ac- general-purpose computing device and at this time celerators like GPUs. OmpSS is based on StartsS a is considered to be in the era of GPUs. (Nickolls & task based programming model. It is based on an- Dally, 2010). However, the main obstacle for large notating a serial application with directives that are adoption on the programmer community has been translated by the compiler. With it, the same pro- the lack of standards that allow programming on gram that runs sequentially in a node with a single unified form different existing hardware solutions GPU can run in parallel in multiple GPUs either (Nickolls & Dally, 2010). The most important play- local (single node) or remote (cluster of GPUs). Be- er on GPU solutions is Nvidia ® with the CUDA® sides performing a task-based parallelization, the language programming and his own compiler runtime system moves the data as needed between (nvcc) (Hill & Marty, 2008), with thousands of in- the different nodes and GPUs minimizing the im- stalled solutions and reward on top500 supercom- pact of communication by using affinity schedul- puter list, while the portability is the main problem. ing, caching, and by overlapping communication Some community project and some hardware alli- with the computational task. ance have proposed solutions for resolve this issue. OmpSs is based on the OpenMP programming OmpSS, OpenACC and OpenMPC have emerged model with modifications to its execution and as the most promising solutions (Vetter, 2012) using memory model. It also provides some extensions the OpenMP base model. In the last year, OpenMP for synchronization, data motion and heterogen- board released the version 4 (OpenMP, 2013) appli- eity support. cation program interface with support for external devices (including GPUs and Vector Processors). In 1) Execution model: OmpSs uses a thread-pool this paper we compare the four implementation of execution model instead of the traditional OpenMP Jacobi’s factorization, to show the advantages and fork-join model. The master thread starts the exe- disadvantages of each framework. cution and all other threads cooperate executing the work it creates (whether it is from work sharing or task constructs). Therefore, there is no need for a METHODOLOGY parallel region. Nesting of constructs allows other threads to generate work as well (Figure 1). The frameworks used working as extensions of 2) Memory model: OmpSs assumes that mul- #pragmas of the C languages offering the simplest tiple address spaces may exist. As such shared way to programming without development com- data may reside in memory locations that are not plicate and external elements. In the next section we describe the frameworks and give some imple- directly accessible from some of the computa- mentations examples. In the last part, we show the tional resources. Therefore, all parallel code can pure CUDA kernels implementations. only safely access private data and shared data which has been marked explicitly with our ex- Ompss tended syntax. This assumption is true even for SMP machines as the implementation may real- Ompss (a programming model form Barcelona Su- locate shared data to improve memory accesses percomputer center based on OpenMP and StarSs) (e.g., NUMA). Tecnura • p-ISSN: 0123-921X • e-ISSN: 2248-7638 • Vol. 18 - Special Edition Doctorate • December 2014 • pp. 160-170 [ 161 ] Parallel programming languages on heterogeneous architectures using openmpc, ompss, openacc and openmp Esteban Hernández B., Gerardo de Jesús Montoya Gaviria, Carlos Enrique Montenegro 3) Extensions: with an attached accelerator device, such as a GPU. Function tasks: OmpSs allows to annotate func- Much of a user application executes on the host. tion declarations or definitions Cilk (Durán, Pérez, Compute intensive regions are offloaded to the ac- Ayguadé, Badia & Labarta, 2008), with a task dir- celerator device under control of the host. The de- ective. In this case, any call to the function creates vice executes parallel regions, which typically a new task that will execute the function body. The contain work- sharing loops, or kernels regions, data environment of the task is captured from the which typically contain one or more loops which function arguments. are executed as kernels on the accelerator. Even in Dependency synchronization:
Load more

Recommended publications
  • Other Apis What’S Wrong with Openmp?
    Threaded Programming Other APIs What’s wrong with OpenMP? • OpenMP is designed for programs where you want a fixed number of threads, and you always want the threads to be consuming CPU cycles. – cannot arbitrarily start/stop threads – cannot put threads to sleep and wake them up later • OpenMP is good for programs where each thread is doing (more-or-less) the same thing. • Although OpenMP supports C++, it’s not especially OO friendly – though it is gradually getting better. • OpenMP doesn’t support other popular base languages – e.g. Java, Python What’s wrong with OpenMP? (cont.) Can do this Can do this Can’t do this Threaded programming APIs • Essential features – a way to create threads – a way to wait for a thread to finish its work – a mechanism to support thread private data – some basic synchronisation methods – at least a mutex lock, or atomic operations • Optional features – support for tasks – more synchronisation methods – e.g. condition variables, barriers,... – higher levels of abstraction – e.g. parallel loops, reductions What are the alternatives? • POSIX threads • C++ threads • Intel TBB • Cilk • OpenCL • Java (not an exhaustive list!) POSIX threads • POSIX threads (or Pthreads) is a standard library for shared memory programming without directives. – Part of the ANSI/IEEE 1003.1 standard (1996) • Interface is a C library – no standard Fortran interface – can be used with C++, but not OO friendly • Widely available – even for Windows – typically installed as part of OS – code is pretty portable • Lots of low-level control over behaviour of threads • Lacks a proper memory consistency model Thread forking #include <pthread.h> int pthread_create( pthread_t *thread, const pthread_attr_t *attr, void*(*start_routine, void*), void *arg) • Creates a new thread: – first argument returns a pointer to a thread descriptor.
    [Show full text]
  • Introduction to Openacc 2018 HPC Workshop: Parallel Programming
    Introduction to OpenACC 2018 HPC Workshop: Parallel Programming Alexander B. Pacheco Research Computing July 17 - 18, 2018 CPU vs GPU CPU : consists of a few cores optimized for sequential serial processing GPU : has a massively parallel architecture consisting of thousands of smaller, more efficient cores designed for handling multiple tasks simultaneously GPU enabled applications 2 / 45 CPU vs GPU CPU : consists of a few cores optimized for sequential serial processing GPU : has a massively parallel architecture consisting of thousands of smaller, more efficient cores designed for handling multiple tasks simultaneously GPU enabled applications 2 / 45 CPU vs GPU CPU : consists of a few cores optimized for sequential serial processing GPU : has a massively parallel architecture consisting of thousands of smaller, more efficient cores designed for handling multiple tasks simultaneously GPU enabled applications 2 / 45 CPU vs GPU CPU : consists of a few cores optimized for sequential serial processing GPU : has a massively parallel architecture consisting of thousands of smaller, more efficient cores designed for handling multiple tasks simultaneously GPU enabled applications 2 / 45 3 / 45 Accelerate Application for GPU 4 / 45 GPU Accelerated Libraries 5 / 45 GPU Programming Languages 6 / 45 What is OpenACC? I OpenACC Application Program Interface describes a collection of compiler directive to specify loops and regions of code in standard C, C++ and Fortran to be offloaded from a host CPU to an attached accelerator. I provides portability across operating systems, host CPUs and accelerators History I OpenACC was developed by The Portland Group (PGI), Cray, CAPS and NVIDIA. I PGI, Cray, and CAPs have spent over 2 years developing and shipping commercial compilers that use directives to enable GPU acceleration as core technology.
    [Show full text]
  • Concurrent Cilk: Lazy Promotion from Tasks to Threads in C/C++
    Concurrent Cilk: Lazy Promotion from Tasks to Threads in C/C++ Christopher S. Zakian, Timothy A. K. Zakian Abhishek Kulkarni, Buddhika Chamith, and Ryan R. Newton Indiana University - Bloomington, fczakian, tzakian, adkulkar, budkahaw, [email protected] Abstract. Library and language support for scheduling non-blocking tasks has greatly improved, as have lightweight (user) threading packages. How- ever, there is a significant gap between the two developments. In previous work|and in today's software packages|lightweight thread creation incurs much larger overheads than tasking libraries, even on tasks that end up never blocking. This limitation can be removed. To that end, we describe an extension to the Intel Cilk Plus runtime system, Concurrent Cilk, where tasks are lazily promoted to threads. Concurrent Cilk removes the overhead of thread creation on threads which end up calling no blocking operations, and is the first system to do so for C/C++ with legacy support (standard calling conventions and stack representations). We demonstrate that Concurrent Cilk adds negligible overhead to existing Cilk programs, while its promoted threads remain more efficient than OS threads in terms of context-switch overhead and blocking communication. Further, it enables development of blocking data structures that create non-fork-join dependence graphs|which can expose more parallelism, and better supports data-driven computations waiting on results from remote devices. 1 Introduction Both task-parallelism [1, 11, 13, 15] and lightweight threading [20] libraries have become popular for different kinds of applications. The key difference between a task and a thread is that threads may block|for example when performing IO|and then resume again.
    [Show full text]
  • GPU Computing with Openacc Directives
    Introduction to OpenACC Directives Duncan Poole, NVIDIA Thomas Bradley, NVIDIA GPUs Reaching Broader Set of Developers 1,000,000’s CAE CFD Finance Rendering Universities Data Analytics Supercomputing Centers Life Sciences 100,000’s Oil & Gas Defense Weather Research Climate Early Adopters Plasma Physics 2004 Present Time 3 Ways to Accelerate Applications Applications OpenACC Programming Libraries Directives Languages “Drop-in” Easily Accelerate Maximum Acceleration Applications Flexibility 3 Ways to Accelerate Applications Applications OpenACC Programming Libraries Directives Languages CUDA Libraries are interoperable with OpenACC “Drop-in” Easily Accelerate Maximum Acceleration Applications Flexibility 3 Ways to Accelerate Applications Applications OpenACC Programming Libraries Directives Languages CUDA Languages are interoperable with OpenACC, “Drop-in” Easily Accelerate too! Maximum Acceleration Applications Flexibility NVIDIA cuBLAS NVIDIA cuRAND NVIDIA cuSPARSE NVIDIA NPP Vector Signal GPU Accelerated Matrix Algebra on Image Processing Linear Algebra GPU and Multicore NVIDIA cuFFT Building-block Sparse Linear C++ STL Features IMSL Library Algorithms for CUDA Algebra for CUDA GPU Accelerated Libraries “Drop-in” Acceleration for Your Applications OpenACC Directives CPU GPU Simple Compiler hints Program myscience Compiler Parallelizes code ... serial code ... !$acc kernels do k = 1,n1 do i = 1,n2 OpenACC ... parallel code ... Compiler Works on many-core GPUs & enddo enddo Hint !$acc end kernels multicore CPUs ... End Program myscience
    [Show full text]
  • Parallelism in Cilk Plus
    Cilk Plus: Language Support for Thread and Vector Parallelism Arch D. Robison Intel Sr. Principal Engineer Outline Motivation for Intel® Cilk Plus SIMD notations Fork-Join notations Karatsuba multiplication example GCC branch Copyright© 2012, Intel Corporation. All rights reserved. 2 *Other brands and names are the property of their respective owners. Multi-Threading and Vectorization are Essential to Performance Latest Intel® Xeon® chip: 8 cores 2 independent threads per core 8-lane (single prec.) vector unit per thread = 128-fold potential for single socket Intel® Many Integrated Core Architecture >50 cores (KNC) ? independent threads per core 16-lane (single prec.) vector unit per thread = parallel heaven Copyright© 2012, Intel Corporation. All rights reserved. 3 *Other brands and names are the property of their respective owners. Importance of Abstraction Software outlives hardware. Recompiling is easier than rewriting. Coding too closely to hardware du jour makes moving to new hardware expensive. C++ philosophy: abstraction with minimal penalty Do not expect compiler to be clever. But let it do tedious bookkeeping. Copyright© 2012, Intel Corporation. All rights reserved. 4 *Other brands and names are the property of their respective owners. “Three Layer Cake” Abstraction Message Passing exploit multiple nodes Fork-Join exploit multiple cores exploit parallelism at multiple algorithmic levels SIMD exploit vector hardware Copyright© 2012, Intel Corporation. All rights reserved. 5 *Other brands and names are the property of their respective owners. Composition Message Driven compose via send/receive Fork-Join compose via call/return SIMD compose sequentially Copyright© 2012, Intel Corporation. All rights reserved. 6 *Other brands and names are the property of their respective owners.
    [Show full text]
  • Openacc Course October 2017. Lecture 1 Q&As
    OpenACC Course October 2017. Lecture 1 Q&As. Question Response I am currently working on accelerating The GCC compilers are lagging behind the PGI compilers in terms of code compiled in gcc, in your OpenACC feature implementations so I'd recommend that you use the PGI experience should I grab the gcc-7 or compilers. PGI compilers provide community version which is free and you try to compile the code with the pgi-c ? can use it to compile OpenACC codes. New to OpenACC. Other than the PGI You can find all the supported compilers here, compiler, what other compilers support https://www.openacc.org/tools OpenACC? Is it supporting Nvidia GPU on TK1? I Currently there isn't an OpenACC implementation that supports ARM think, it must be processors including TK1. PGI is considering adding this support sometime in the future, but for now, you'll need to use CUDA. OpenACC directives work with intel Xeon Phi is treated as a MultiCore x86 CPU. With PGI you can use the "- xeon phi platform? ta=multicore" flag to target multicore CPUs when using OpenACC. Does it have any application in field of In my opinion OpenACC enables good software engineering practices by Software Engineering? enabling you to write a single source code, which is more maintainable than having to maintain different code bases for CPUs, GPUs, whatever. Does the CPU comparisons include I think that the CPU comparisons include standard vectorisation that the simd vectorizations? PGI compiler applies, but no specific hand-coded vectorisation optimisations or intrinsic work. Does OpenMP also enable OpenMP does have support for GPUs, but the compilers are just now parallelization on GPU? becoming available.
    [Show full text]
  • A CPU/GPU Task-Parallel Runtime with Explicit Epoch Synchronization
    TREES: A CPU/GPU Task-Parallel Runtime with Explicit Epoch Synchronization Blake A. Hechtman, Andrew D. Hilton, and Daniel J. Sorin Department of Electrical and Computer Engineering Duke University Abstract —We have developed a task-parallel runtime targeting CPUs are a poor fit for GPUs. To understand system, called TREES, that is designed for high why this mismatch exists, we must first understand the performance on CPU/GPU platforms. On platforms performance of an idealized task-parallel application with multiple CPUs, Cilk’s “work-first” principle (with no runtime) and then how the runtime’s overhead underlies how task-parallel applications can achieve affects it. The performance of a task-parallel application performance, but work-first is a poor fit for GPUs. We is a function of two characteristics: its total amount of build upon work-first to create the “work-together” work to be performed (T1, the time to execute on 1 principle that addresses the specific strengths and processor) and its critical path (T∞, the time to execute weaknesses of GPUs. The work-together principle on an infinite number of processors). Prior work has extends work-first by stating that (a) the overhead on shown that the runtime of a system with P processors, the critical path should be paid by the entire system at TP, is bounded by = ( ) + ( ) due to the once and (b) work overheads should be paid co- greedy o ff line scheduler bound [3][10]. operatively. We have implemented the TREES runtime A task-parallel runtime introduces overheads and, for in OpenCL, and we experimentally evaluate TREES purposes of performance analysis, we distinguish applications on a CPU/GPU platform.
    [Show full text]
  • Multi-Threaded GPU Accelerration of ORBIT with Minimal Code
    Princeton Plasma Physics Laboratory PPPL- 4996 4996 Multi-threaded GPU Acceleration of ORBIT with Minimal Code Modifications Ante Qu, Stephane Ethier, Eliot Feibush and Roscoe White FEBRUARY, 2014 Prepared for the U.S. Department of Energy under Contract DE-AC02-09CH11466. Princeton Plasma Physics Laboratory Report Disclaimers Full Legal Disclaimer This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, nor any of their contractors, subcontractors or their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or any third party’s use or the results of such use of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof or its contractors or subcontractors. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof. Trademark Disclaimer Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof or its contractors or subcontractors. PPPL Report Availability Princeton Plasma Physics Laboratory: http://www.pppl.gov/techreports.cfm Office of Scientific and Technical Information (OSTI): http://www.osti.gov/bridge Related Links: U.S.
    [Show full text]
  • Intel Threading Building Blocks
    Praise for Intel Threading Building Blocks “The Age of Serial Computing is over. With the advent of multi-core processors, parallel- computing technology that was once relegated to universities and research labs is now emerging as mainstream. Intel Threading Building Blocks updates and greatly expands the ‘work-stealing’ technology pioneered by the MIT Cilk system of 15 years ago, providing a modern industrial-strength C++ library for concurrent programming. “Not only does this book offer an excellent introduction to the library, it furnishes novices and experts alike with a clear and accessible discussion of the complexities of concurrency.” — Charles E. Leiserson, MIT Computer Science and Artificial Intelligence Laboratory “We used to say make it right, then make it fast. We can’t do that anymore. TBB lets us design for correctness and speed up front for Maya. This book shows you how to extract the most benefit from using TBB in your code.” — Martin Watt, Senior Software Engineer, Autodesk “TBB promises to change how parallel programming is done in C++. This book will be extremely useful to any C++ programmer. With this book, James achieves two important goals: • Presents an excellent introduction to parallel programming, illustrating the most com- mon parallel programming patterns and the forces governing their use. • Documents the Threading Building Blocks C++ library—a library that provides generic algorithms for these patterns. “TBB incorporates many of the best ideas that researchers in object-oriented parallel computing developed in the last two decades.” — Marc Snir, Head of the Computer Science Department, University of Illinois at Urbana-Champaign “This book was my first introduction to Intel Threading Building Blocks.
    [Show full text]
  • An Overview of Parallel Ccomputing
    An Overview of Parallel Ccomputing Marc Moreno Maza University of Western Ontario, London, Ontario (Canada) CS2101 Plan 1 Hardware 2 Types of Parallelism 3 Concurrency Platforms: Three Examples Cilk CUDA MPI Hardware Plan 1 Hardware 2 Types of Parallelism 3 Concurrency Platforms: Three Examples Cilk CUDA MPI Hardware von Neumann Architecture In 1945, the Hungarian mathematician John von Neumann proposed the above organization for hardware computers. The Control Unit fetches instructions/data from memory, decodes the instructions and then sequentially coordinates operations to accomplish the programmed task. The Arithmetic Unit performs basic arithmetic operation, while Input/Output is the interface to the human operator. Hardware von Neumann Architecture The Pentium Family. Hardware Parallel computer hardware Most computers today (including tablets, smartphones, etc.) are equipped with several processing units (control+arithmetic units). Various characteristics determine the types of computations: shared memory vs distributed memory, single-core processors vs multicore processors, data-centric parallelism vs task-centric parallelism. Historically, shared memory machines have been classified as UMA and NUMA, based upon memory access times. Hardware Uniform memory access (UMA) Identical processors, equal access and access times to memory. In the presence of cache memories, cache coherency is accomplished at the hardware level: if one processor updates a location in shared memory, then all the other processors know about the update. UMA architectures were first represented by Symmetric Multiprocessor (SMP) machines. Multicore processors follow the same architecture and, in addition, integrate the cores onto a single circuit die. Hardware Non-uniform memory access (NUMA) Often made by physically linking two or more SMPs (or multicore processors).
    [Show full text]
  • Openclunk: a Cilk-Like Framework for the GPGPU
    OpenCLunk: a Cilk-like framework for the GPGPU Lena Olson, Markus Peloquin {lena, markus}@cs December 20, 2010 Abstract models, tools, and languages that exist to exploit parallelism on CPUs. One example is the task-based The potential for GPUs to attain high speedup parallel language Cilk. It uses a simple but powerful over traditional multicore systems has made them extension of C to allow programmers to easily write a popular target for parallel programmers. However, code. they are notoriously difficult to program and debug. We decided to try to combine the elegance of Cilk Meanwhile, there are several programming models in with the acceleration of the GPU. From the start, the multicore domain which are designed to be sim- we knew that this would not be simple. Cilk is de- ple to design and program for, such as Cilk. Open- signed to be a task-parallel language, and GPUs ex- CLunk attempts to combine Cilk’s ease of program- cel at data parallelism. However, we thought that it ming with the good performance of the GPU. We might be advantageous to have a model that was sim- implemented a Cilk-like task-parallel programming ple to program, even if the speedup was more mod- framework for the GPU, and evaluated its perfor- est than the exceptional cases often cited. Therefore, mance. The results indicate that while it is possible we had several goals. The first was to simply imple- to implement such a system, it is not possible to ment a Cilk-like model, where Cilk programs could attain good performance, and therefore a Cilk-like be easily ported and run: something simple enough system is currently not feasible as a programming for a compiler to target.
    [Show full text]
  • Pragma Omp Parallel Do Something Else();
    CS 470 Spring 2018 Mike Lam, Professor OpenMP OpenMP ● Programming language extension – Compiler support required – "Open Multi-Processing" (open standard; latest version is 4.5) ● “Automatic” thread-level parallelism – Guided by programmer-supplied directives – Does NOT verify correctness of parallel transformations – Targets shared-memory systems – Used in distributed systems for on-node parallelism ● Other similar techs: Cilk, OpenACC – OpenMP is currently the most popular CPU-based technology Fork-join threading ● OpenMP provides directives to control threading – General fork-join threading model w/ teams of threads – One master thread and multiple worker threads Source: https://en.wikipedia.org/wiki/Fork–join_model C preprocessor ● Text-based processing phase of compilation – Can be run individually with “cpp” ● Controlled by directives on lines beginning with “#” – Must be the first non-whitespace character – Alignment is a matter of personal style #include <stdio.h> #include <stdio.h> #define FOO #define FOO #define BAR 5 #define BAR 5 int main() { int main() { # ifdef FOO #ifdef FOO printf("Hello!\n"); printf("Hello!\n"); # else #else printf("Goodbye!\n"); printf("Goodbye!\n"); # endif #endif printf("%d\n", BAR); printf("%d\n", BAR); return 0; return 0; } } my preference Pragmas ● #pragma - generic preprocessor directive – Provides direction or info to later compiler phases – Ignored by compilers that don't support it – All OpenMP pragma directives begin with "omp" – Basic threading directive: "parallel" ● Runs the following code
    [Show full text]