Cluster Programming using the OpenMP Accelerator Model Hervé Yviquel, Lauro Cruz, Guido Araújo

To cite this version:

Hervé Yviquel, Lauro Cruz, Guido Araújo. Cluster Programming using the OpenMP Accelerator Model. ACM Transactions on Architecture and Code Optimization, Association for Computing Ma- chinery, 2018, 15 (3), pp.1-23. ￿10.1145/3226112￿. ￿hal-01984327￿

HAL Id: hal-01984327 https://hal.archives-ouvertes.fr/hal-01984327 Submitted on 16 Jan 2019

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. 1

Cluster Programming using the OpenMP Accelerator Model

HERVÉ YVIQUEL, Institute of Computing, University of Campinas (UNICAMP), Brazil LAURO CRUZ, Institute of Computing, University of Campinas (UNICAMP), Brazil GUIDO ARAUJO, Institute of Computing, University of Campinas (UNICAMP), Brazil is a programming model in which program fragments (e.g. hot loops) are annotated so that their execution is performed in dedicated hardware or accelerator devices. Although offloading has been extensively used to move computation to GPUs, through directive-based annotation standards like OpenMP, offloading computation to very large clusters can become a complex and cumbersome task. It typically requires mixing programming models (e.g. OpenMP and MPI) and languages (e.g. /C++ and Scala), dealing with various access control mechanisms from different cloud providers (e.g. AWS and Azure), and integrating all this into a single application. This paper introduces computer cluster nodes as simple OpenMP offloading devices that can be used either from a local computer or from the cluster head-. Itproposesa methodology that transforms OpenMP directives to Spark runtime calls with fully integrated communication management, in a way that a cluster appears to the programmer as yet another accelerator device. Experiments using LLVM 3.8, OpenMP 4.5 on well known cloud infrastructures ( Azure and Amazon EC2) show the viability of the proposed approach, enable a thorough analysis of its performance and make a comparison with an MPI implementation. The results show that although data transfers can impose overheads, cloud offloading from a local machine can still achieve promising speedups for larger granularity: upto115xin 256 cores for the 2MM benchmark using 1GB sparse matrices. In addition, the parallel implementation of a complex and relevant scientific application reveals a 80x on a 320 cores machine when executed directly from the headnode of the cluster. CCS Concepts: • Computing methodologies → Distributed programming languages; • Software and its engineering → Parallel programming languages; Distributed programming languages; Runtime environments; Source code generation; Additional Key Words and Phrases: OpenMP, LLVM, ACM Reference Format: Hervé Yviquel, Lauro Cruz, and Guido Araujo. 2018. Cluster Programming using the OpenMP Accelerator Model. ACM Trans. Arch. Code Optim. 1, 1, Article 1 (January 2018), 24 pages. https://doi.org/10.1145/3226112

1 INTRODUCTION

Parallelizing loops is a well-known research problem that has been extensively studied. Most of the approaches to this problem use