DOCSLIB.ORG

(12) United States Patent (10) Patent No.: US 8.209,664 B2 Yu Et Al

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
(12) United States Patent (10) Patent No.: US 8.209,664 B2 Yu Et Al USOO8209.664B2 (12) United States Patent (10) Patent No.: US 8.209,664 B2 Yu et al. (45) Date of Patent: Jun. 26, 2012 (54) HIGH LEVEL PROGRAMMING 2008, OO86442 A1 4/2008 Dasdan et al. 2008.OO98375 A1 4/2008 Isard EXTENSIONS FOR DISTRIBUTED DATA 2008/0120314 A1 5/2008 Yang et al. PARALLEL PROCESSING 2008/O127200 A1 5/2008 Richards et al. 2008/0250227 A1 10, 2008 Linderman et al. (75) Inventors: Yuan Yu, Cupertino, CA (US); Ulfar 2009/0043803 A1* 2/2009 Frishberg et al. ............. 707/102 Erlingsson, Reykjavik (IS); Michael A 2009/0300656 A1* 12/2009 Bosworth et al. ............. T19,320 Isard, San Francisco, CA (US); Frank OTHER PUBLICATIONS McSherry, San Francisco, CA (US) Blelloch, Guy E., et al., “Implementation of a Portable Nested Data (73) Assignee: Microsoft Corporation, Redmond, WA Parallel Language.” School of Computer Science, Carnegie Mellon (US) University, Pittsburgh, PA, Feb. 1993, Abstract, pp. 1-28. Isard, Michael, et al., “Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks.” Microsoft Research, Sillicon Val (*) Notice: Subject to any disclaimer, the term of this ley, http://research.microsoft.com/research/dv/dryad/eurosysO7. patent is extended or adjusted under 35 pdf). Mar. 23, 2007, 14 pages. U.S.C. 154(b) by 704 days. Ferreira, Renato, et al., “Data parallel language and compiler Support for data intensive applications*1.” Elsevier Science B.V., Parallel (21) Appl. No.: 12/406,826 Computing, vol. 28, Issue 5, May, 2002, 3 pages. Carpenter, Bryan, et al., “HPJava: Data Parallel Extensions to Java.” (22) Filed: Mar 18, 2009 NPAC at Syracuse University, Syracuse, NY, Feb. 7, 1998, pp. 1-5. (65) Prior Publication Data (Continued) US 2010/O241827 A1 Sep. 23, 2010 Primary Examiner — Anna Deng (74) Attorney, Agent, or Firm — Vierra Magen Marcus & (51) Int. C. DeNiro LLP G06F 9/44 (2006.01) (52) U.S. Cl. ....................................................... T17/119 (57) ABSTRACT (58) Field of Classification Search .................. 717/106, General-purpose distributed data-parallel computing using 717/109, 114, 119 high-level computing languages is described. Data parallel See application file for complete search history. portions of a sequential program that is written by a developer (56) References Cited in a high-level language are automatically translated into a distributed execution plan. A set of extensions to a sequential U.S. PATENT DOCUMENTS high-level computing language are provided to Support dis 7,802,180 B2 * 9/2010 Warner et al. ................. T15,234 tributed parallel computations and to facilitate generation and 2005/0177585 A1 8/2005 Venkatesh et al. ............ 707/100 optimization of distributed execution plans. The extensions 2006/0116578 A1* 6/2006 Grunwald et al. ............ 600/440 are fully integrated with the programming language, thereby 2007/OO 11155 A1 1/2007 Sarkar enabling developers to write sequential language programs 2007/0O386.59 A1 2/2007 Datar et al. 2007/0043531 A1 2/2007 Kosche et al. ................ TO2, 182 using known constructs while providing the ability to invoke 2008.0005547 A1 1/2008 Papakipos et al. the extensions to enable better generation and optimization of 2008.0034.357 A1 2/2008 GSchwind the execution plan for a distributed computing environment. 2008.OO79724 A1 4/2008 Isard et al. 2008/0082644 A1 4/2008 Isard et al. 17 Claims, 18 Drawing Sheets 508 receive user-defined function with Apply operator homomorphic? generate vertex code to generate function vertex stream all dataset partitions code for all nodes having 512 to single node dataset partition generate function vertex Code for single node appying function US 8,209,664 B2 Page 2 OTHER PUBLICATIONS Catanzaro, Bryan, et al., “A Map Reduce Framework for Program ming Graphics Processors.” University of California, Berkeley, CA, Duffy, Joe, et al., “Parallel LINQ Running Queries on Multi-Core http://web.mit.edu/rabbah/www.conferences/08/stmcspapers/ Processors.” Microsoft Corporation, http://msdn.microsoft.com/en catanzaro-stmcs08.pdf, 2008, 6 pages. us/magazine/cc 163329.aspx. 2008, 8 pages. Dean, Jeffrey, et al., “MapReduce: Simplified Data Processing on Yu, Yuan, et al., “DryadLINQ: A System for General-Purpose Dis Large Clusters. Google, Inc. http://labs.google.com/papers/ tributed Data-Parallel Computing Using a High-Level Language.” 8th Usenix Symposium on Operating Systems Design and Imple mapreduce-osdi O4.pdf. OSDI 2004, pp. 1-13. mentation, Dec. 8, 2008, pp. 1-14. Olston, Christopher, et al., “Automatic Optimization of Parallel Yu, Yuan, et al., “Some sample programs written in DryadLINQ.” Dataflow Programs.” Yahoo! Research, http://www.usenix.org/ Microsoft Corporation, ftp://ftp.research.microsoft.com/pub?tr/TR event/usenix08/tech/full papers?olston?olston html/index.html). 2008-74.pdf, May 11, 2008, pp. 1-37. Apr. 29, 2008, 11 pages. Isard, Michael, Dryad and DryadLINQ, Presentation to Hadoop U.S. Appl. No. 12/368,231, filed Feb. 9, 2009. Summit, Microsoft Corporation, Mar. 25, 2008, 15 pages. Isard, Michael, et al., “Simple programming for simple distributed systems.” Cambridge SRG, Jun. 9, 2005. * cited by examiner U.S. Patent US 8,209,664 B2 ??un61-I U.S. Patent Jun. 26, 2012 Sheet 2 of 18 US 8,209,664 B2 Figure 2 106 100 108 110 116 114 Input Device(s) 112 Communication 17 Connection(s) 152 154 156 158 160 162 170 172 174 176 178 18O U.S. Patent Jun. 26, 2012 Sheet 3 of 18 US 8,209,664 B2 Figure 4 228 Generate execution plan graph 4O2 Perform static optimizations 404 4 O Generate COce and Static data for Vertices 4O6 Generate Serialized Objects and Serialization- 408 Code for required data types Generate Code for dynamic optimizations 410 Figure 8 U.S. Patent Jun. 26, 2012 Sheet 4 of 18 US 8,209,664 B2 Figure 5 create and build graph based on EPG 242 receive list Of nodes from Name Server 244 determine Which nodes are available 246 populate available nodes into Node Queue 248 populate vertices into Vertex Queue 250 identify vertices that are ready 252 Send instructions to available nodes to 254 execute vertices that are ready Send COOle to available nodes for Vertices that are ready, if not cached (on same 256 machine or locally) 258 260 manage Node Queue manage Vertex Queue U.S. Patent Jun. 26, 2012 Sheet 5 of 18 US 8,209,664 B2 Developer Application Program 310 Expression 312 LOCal Table Objects Distributed Eston Provider 324 Serialized DataObjects Execution and Serialization COce Plan Graph 316 318 Distributed Compute System 304 Compute System Output Tables 322 Figure 6 U.S. Patent Jun. 26, 2012 Sheet 6 of 18 US 8,209,664 B2 099 Je6eue.Wqof Z?un61-I 899 U.S. Patent Jun. 26, 2012 Sheet 7 of 18 US 8,209,664 B2 U.S. Patent Jun. 26, 2012 Sheet 8 of 18 US 8,209,664 B2 receive expression having Figure 10 input, intermediate result or 452 dataset expression include user-Supplied partitioning operator? 458 add default dynamic add user-supplied data partitioning operator to partitioning operator to graph graph Continue compiling graph U.S. Patent Jun. 26, 2012 Sheet 9 of 18 US 8,209,664 B2 470 public Oueryable.<TSource> Hash Partition<TSource, TKey>(Queryable-TSource> source, Expression<Func-TSource, TKey>> keySelector, EqualityComparer(TKey> Comparer, int Count) Figure 11A 472 public Queryable:CTSource> HashPartition<TSource, TKey>(Queryable.<TSource> source, Expression<Func<TSource, TKey>> keySelector, |EqualityComparerCTKey> Comparer) Figure 11B 474 public static Queryable.<TSource> RangePartition<TSource, TKey>(IQueryable.<TSource> source, Expression<FunczTSource, TKey>> keySelector, TKey rangeKeys, ComparerCTKey> Comparer) Figure 12A 476 public static Queryable(TSource> RangePartition<TSource, TKey>(IQueryable CTSource> source, Expression<Func-TSource, TKey>> keySelector, |ComparerCTKey> Comparer) Figure 12B U.S. Patent Jun. 26, 2012 Sheet 10 of 18 US 8,209,664 B2 480 public static MultiQueryable.<R1, R22 Fork<T, R1, R2-(Queryable<T> source, Expression<Funcg|Enumerable-T2, IEnumerable<ForkTuple.<R1,R2>>>> mapper) Figure 13A 482 public static MultiQueryable.<R1, R2, R32 Fork<T, R1, R2, R32(IQueryable-Ta source, Expression<Funcg|Enumerable-Ta, IEnumerable, ForkTuple.<R1, R2, R322>> mapper) Figure 13B 484 public static Keyed MultiQueryable.<T, K-> Fork<T, K>(IQueryableCT> source, Expression<Func-T, KY keySelector, KD keys) Figure 13C 498 war q = source. Fork(p => new ForkTuple(int, inte(p.X, p.y)); war q1 = q. First, war q2 = q. Second; Figure 15 U.S. Patent Jun. 26, 2012 Sheet 11 of 18 US 8,209,664 B2 access Source dataset 490 Figure 14 access user-defined mapper function (or keyselector function 492 and keys) map records of type T in source dataset into different datastreams of 494 tuples of type R1, R2, etc. return object with embedded 496 datastreams 508 receive USer-defined function with Apply operator Figure 17 homomorphic? generate vertex Code to generate function vertex stream all dataset partitions Code for all nodes having 512 to single node dataset partition generate function vertex code for single node appying function U.S. Patent Jun. 26, 2012 Sheet 12 of 18 US 8,209,664 B2 502 public static IQueryable.<T2> Apply<T1, T2>(Queryable.<T1> source, Expression<Funcg.IEnumerable CT12, IEnumerable CT2>>> procFunc) Figure 16A 504 public static IQueryable.<T3> Apply<T1, T2, T3>(Queryable<T1> source1, Queryable-T2> source2, Expression<Funcg|Enumerable.<T1>, Enumerable.<T2>, IEnumerable<T3>>> procFunc) Figure 16B 506 public static IEnumerable.<double> SlidingAverage(IEnumerable.<int> source) { int window = new int10); int i = 0; foreach (int x in source) { windowi) = x; i = (i+1) %10; yield return (window. Sum()/10.0); } Figure 16C U.S. Patent Jun. 26, 2012 Sheet 13 of 18 US 8,209,664 B2 520 Homomorphic public static Enumerable-int> addeach(IEnumerable.<int> left, Enumerable.<int> right) { IEnumerator-int> left enu = left.GetEnumerator(); IEnumerator-int> right enu = right.GetEnumerator(); while (true) { bool more left = left enu.
Load more

Recommended publications
  • Technical Report Aaron Councilman
    Extensible Parallel Programming in ableC Aaron Councilman Department of Computer Science and Engineering University of Minnesota, Twin Cities May 23, 2019 1 Introduction There are many different manners of parallelizing code, and many different languages that provide such features. Different types of computations are best suited by different types of parallelism. Simply whether a computation is compute bound or I/O bound determines whether the computation will benefit from being run with more threads than the machine has cores, and other properties of a computation will similarly affect how it performs when run in parallel. Thus, to provide parallel programmers the ability to deliver the best performance for their programs, the ability to choose the parallel programming abstractions they use is important. The ability to combine these abstractions however they need is also important, since different parts of a program will have different performance properties, and therefore may perform best using different abstractions. Unfortunately, parallel programming languages are often designed monolithically, built as an entire language with a specific set of features. Because of this, programmer's choice of parallel programming abstractions is generally limited to the choice of the language to use. Beyond limiting the available abstracts, this also means, that the choice of abstractions must be made ahead of time, since any attempt to change the parallel programming language at a later time is likely to be be prohibitive as it may require rewriting large portions of the codebase, if not the entire codebase. Extensible programming languages can offer a solution to these problems. With an extensible compiler, the programmer chooses a base programming language and can then select the set of \extensions" for that language that best fit their needs.
    [Show full text]
  • Intel® Cilk™ Plus
    Overview: Programming Environment for Intel® Xeon Phi™ Coprocessor One Source Base, Tuned to many Targets Source Compilers, Libraries, Parallel Models Multicore Many-core Cluster Multicore Multicore CPU CPU Intel® MIC Multicore Multicore and Architecture Cluster Many-core Cluster Copyright© 2014, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners. Intel® Parallel Studio XE 2013 and Intel® Cluster Studio XE 2013 Phase Product Feature Benefit Intel® Threading design assistant • Simplifies, demystifies, and speeds Advisor XE (Studio products only) parallel application design • C/C++ and Fortran compilers • Intel® Threading Building Blocks • Enabling solution to achieve the Intel® • Intel® Cilk™ Plus application performance and Composer XE • Intel® Integrated Performance scalability benefits of multicore and Build Primitives forward scale to many-core • Intel® Math Kernel Library • Enabling High Performance Scalability, Interconnect Intel® High Performance Message Independence, Runtime Fabric † MPI Library Passing (MPI) Library Selection, and Application Tuning Capability ® Intel Performance Profiler for • Remove guesswork, saves time, VTune™ optimizing application makes it easier to find performance Amplifier XE performance and scalability and scalability bottlenecks Memory & threading dynamic • Increased productivity, code quality, ® Intel analysis for code quality and lowers cost, finds memory, Verify Inspector XE threading , and security defects & Tune Static Analysis for code quality
    [Show full text]
  • Part V Some Broad Topic
    Part V Some Broad Topic Winter 2021 Parallel Processing, Some Broad Topics Slide 1 About This Presentation This presentation is intended to support the use of the textbook Introduction to Parallel Processing: Algorithms and Architectures (Plenum Press, 1999, ISBN 0-306-45970-1). It was prepared by the author in connection with teaching the graduate-level course ECE 254B: Advanced Computer Architecture: Parallel Processing, at the University of California, Santa Barbara. Instructors can use these slides in classroom teaching and for other educational purposes. Any other use is strictly prohibited. © Behrooz Parhami Edition Released Revised Revised Revised First Spring 2005 Spring 2006 Fall 2008 Fall 2010 Winter 2013 Winter 2014 Winter 2016 Winter 2019* Winter 2021* *Chapters 17-18 only Winter 2021 Parallel Processing, Some Broad Topics Slide 2 V Some Broad Topics Study topics that cut across all architectural classes: • Mapping computations onto processors (scheduling) • Ensuring that I/O can keep up with other subsystems • Storage, system, software, and reliability issues Topics in This Part Chapter 17 Emulation and Scheduling Chapter 18 Data Storage, Input, and Output Chapter 19 Reliable Parallel Processing Chapter 20 System and Software Issues Winter 2021 Parallel Processing, Some Broad Topics Slide 3 17 Emulation and Scheduling Mapping an architecture or task system onto an architecture • Learn how to achieve algorithm portability via emulation • Become familiar with task scheduling in parallel systems Topics in This Chapter 17.1 Emulations Among Architectures 17.2 Distributed Shared Memory 17.3 The Task Scheduling Problem 17.4 A Class of Scheduling Algorithms 17.5 Some Useful Bounds for Scheduling 17.6 Load Balancing and Dataflow Systems Winter 2021 Parallel Processing, Some Broad Topics Slide 4 17.1 Emulations Among Architectures Need for scheduling: a.
    [Show full text]
  • An Introduction to Parallel Programming
    In Praise of An Introduction to Parallel Programming With the coming of multicore processors and the cloud, parallel computing is most cer- tainly not a niche area off in a corner of the computing world. Parallelism has become central to the efficient use of resources, and this new textbook by Peter Pacheco will go a long way toward introducing students early in their academic careers to both the art and practice of parallel computing. Duncan Buell Department of Computer Science and Engineering University of South Carolina An Introduction to Parallel Programming illustrates fundamental programming principles in the increasingly important area of shared-memory programming using Pthreads and OpenMP and distributed-memory programming using MPI. More important, it empha- sizes good programming practices by indicating potential performance pitfalls. These topics are presented in the context of a variety of disciplines, including computer science, physics, and mathematics. The chapters include numerous programming exercises that range from easy to very challenging. This is an ideal book for students or professionals looking to learn parallel programming skills or to refresh their knowledge. Leigh Little Department of Computational Science The College at Brockport, The State University of New York An Introduction to Parallel Programming is a well-written, comprehensive book on the field of parallel computing. Students and practitioners alike will appreciate the rele- vant, up-to-date information. Peter Pacheco’s very accessible writing style, combined with numerous interesting examples, keeps the reader’s attention. In a field that races forward at a dizzying pace, this book hangs on for the wild ride covering the ins and outs of parallel hardware and software.
    [Show full text]
  • Lecture Notes : Intel Xeon Phi Coprocessor - an Overview
    C-DAC Four Days Technology Workshop ON Hybrid Computing – Coprocessors/Accelerators Power-Aware Computing – Performance of Application Kernels hyPACK-2013 Mode 3 : Intel Xeon Phi Coprocessors Lecture Notes : Intel Xeon Phi Coprocessor - An Overview Venue : CMSD, UoHYD ; Date : October 15-18, 2013 C-DAC hyPACK-2013 Xeon-Phi Coprocessors : An Overview 1 An Overview of Prog. Env on Intel Xeon-Phi Lecture Outline Following topics will be discussed Understanding of Intel Xeon-Phi Coprocessor Architecture Programming on Intel Xeon-Phi Coprocessor Performance Issues on Intel Xeon-Phi Coprocessor C-DAC hyPACK-2013 Xeon-Phi Coprocessors : An Overview 2 Intel Xeon Host : An Overview of Xeon - Multi-Core and Systems with Devices Part-I Background : Xeon Host - Multi-Core & Devices C-DAC hyPACK-2013 Xeon-Phi Coprocessors : An Overview 3 Programming paradigms-Challenges Large scale data Computing – Current trends HowTrends to run Computing Programs - Observationsfaster ? You require Super Computer Era of Single - Multi-to-Many Core - Heterogeneous Computing Sequential Computing How to run Programs faster ? Fetch/Store Fast Access of data Compute Fast Processor More Memory to Manage data C-DAC hyPACK-2013 Xeon-Phi Coprocessors : An Overview 4 Multi-threaded Processing using Hyper-Threading Technology Time taken to process n threads on a single processor is significantly more than a single processor system with HT technology enabled. Multithreading Hyper-threading Technology App 0 App 1 App 2 App 0 App 1 App 2 T0 T1 T2 T3 T4 T5 T0 T1 T2 T3 T4 T5 Thread Pool Thread Pool CPU CPU CPU T0 T1 T2 T3 T4 T5 LP0 T0 T2 T4 LP1 Time T1 T3 T5 CPU 2 Threads per Processor Time Source : http://www.intel.com ; Reference : [6], [29], [31] P/C : Microprocessor and cache; SM : Shared memory C-DAC hyPACK-2013 Xeon-Phi Coprocessors : An Overview 5 Relationship among Processors, Processes, &Threads Processors Processors µPi Processes .
    [Show full text]
  • The Relevance of Opencl to HPC
    The Relevance of OpenCL to HPC . Paul Preney, OCT, M.Sc., B.Ed., B.Sc. [email protected] School of Computer Science University of Windsor Windsor, Ontario, Canada Copyright © 2015 Paul Preney. All Rights Reserved. March 4, 2015 The Relevance of OpenCL to HPC Paul Preney 2 Announced Abstract Today’s high performance computing programs are evolving to be increas- ingly more parallel, increasingly more wait-free, increasingly deployed on many different kinds of hardware including general purpose CPUs, GPGPUs, FPGAs, other custom specific-purpose hardware, etc. The OpenCL standards are platforms providing interfaces that enable deployment of programs to vir- tually any heterogeneous computing device. The OpenCL standard defines a highly-vectorizable programming language, OpenCL C, which enables the deployment of programming logic to arbitrary hardware without requiring low-level, “machine-coding” knowledge of such. The OpenCL standard is a critical component of exascale initiatives given that it is hardware neu- tral, with significant support and participation from all the major processor vendors. Unfortunately the main source of information about OpenCL is in the form of its final specifications so there is a lot of misinformation about it. This talk will explain the relevance of the OpenCL standard to the HPC community, and offer a glimpse into what high-level abstractions for OpenCL, under development by software engineers, might look like. The Relevance of OpenCL to HPC Paul Preney 3 Presentation Overview 1. HPC and OpenCL 2. The Relevance Of OpenCL To HPC 3. A Glimpse Into Possible High-Level Abstractions 4. The Future and Discussion 5. References The Relevance of OpenCL to HPC Paul Preney 4 Table of Contents 1.
    [Show full text]
  • Speculative Parallelism in Cilk++
    Speculative Parallelism in Cilk++ Ruben Perez Gregory Malecha MIT Harvard University SEAS [email protected] [email protected] ABSTRACT and are based around backtracking search, often optimized Backtracking search algorithms are useful in many domains, with heuristics. Algorithms such as these can benefit tremen- from SAT solvers to artificial intelligences for playing games dously from parallel processing because separate threads can such as chess. Searching disjoint branches can, inherently be used to search independent paths in parallel. We call be done in parallel though it can considerably increase the this strategy speculative parallelism because only one solu- amount of work that the algorithm does. Such parallelism is tion matters and as soon as that is found, all other work can speculative, once a solution is found additional work is irrele- be stopped. vant, but the individual branches each have their own poten- tial to find the solution. In sequential algorithms, heuristics While simple to describe, this use of parallelism is not always are used to prune regions of the search space. In parallel easy to implement. One parallel programming platform, implementations this pruning often corresponds to abort- cilk5 [FLR98], included language support for this type of ing existing computations that can be shown to be pursuing parallelism, but it has since been removed in more recent dead-ends. While some systems provide native support for versions of the platform adapted to C++, cilk++ [Lei09]. aborting work, Intel's current parallel extensions to C++, Our goal is to regain the power of speculative execution in a implemented in the cilk++ compiler [Lei09], do not.
    [Show full text]
  • Parallel Programming with Matlabmpi ∗
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by CERN Document Server Parallel Programming with MatlabMPI ∗ Jeremy Kepner ([email protected]) MIT Lincoln Laboratory, Lexington, MA 02420 July 23, 2001 Abstract MatlabMPI is a Matlab implementation of the Message Passing Interface (MPI) standard and allows any Matlab program to exploit multiple processors. MatlabMPI currently implements the basic six functions that are the core of the MPI point-to-point communications standard. The key technical innovation of MatlabMPI is that it implements the widely used MPI \look and feel" on top of standard Matlab file I/O, resulting in an extremely compact ( 100 lines) and \pure" implementation which runs anywhere Matlab runs. The performance has been tested on∼ both shared and distributed memory parallel computers. MatlabMPI can match the bandwidth of C based MPI at large message sizes. A test image filtering application using MatlabMPI achieved a speedup of 70 on a parallel computer. ∼ 1 Introduction Matlab [1] is the dominant programming language for implementing numerical computations and is widely used for algorithm development, simulation, data reduction, testing and system evaluation. Many of these computations can benefit from faster execution on a parallel computer. There have been many previous attempts to provide an efficient mechanism for running Matlab programs on parallel computers [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]. These efforts of have faced numerous challenges and none have received widespread acceptance. In the world of parallel computing the Message Passing Interface (MPI) [2] is the de facto standard for implementing programs on multiple processors.
    [Show full text]
  • Parallel Programming in Openmp About the Authors
    Parallel Programming in OpenMP About the Authors Rohit Chandra is a chief scientist at NARUS, Inc., a provider of internet business infrastructure solutions. He previously was a principal engineer in the Compiler Group at Silicon Graphics, where he helped design and implement OpenMP. Leonardo Dagum works for Silicon Graphics in the Linux Server Platform Group, where he is responsible for the I/O infrastructure in SGI’s scalable Linux server systems. He helped define the OpenMP Fortran API. His research interests include parallel algorithms and performance modeling for parallel systems. Dave Kohr is a member of the technical staff at NARUS, Inc. He previ- ously was a member of the technical staff in the Compiler Group at Silicon Graphics, where he helped define and implement the OpenMP. Dror Maydan is director of software at Tensilica, Inc., a provider of appli- cation-specific processor technology. He previously was an engineering department manager in the Compiler Group of Silicon Graphics, where he helped design and implement OpenMP. Jeff McDonald owns SolidFX, a private software development company. As the engineering department manager at Silicon Graphics, he proposed the OpenMP API effort and helped develop it into the industry standard it is today. Ramesh Menon is a staff engineer at NARUS, Inc. Prior to NARUS, Ramesh was a staff engineer at SGI, representing SGI in the OpenMP forum. He was the founding chairman of the OpenMP Architecture Review Board (ARB) and supervised the writing of the first OpenMP specifica- tions. Parallel Programming in OpenMP Rohit Chandra Leonardo Dagum Dave Kohr Dror Maydan Jeff McDonald Ramesh Menon Senior Editor Denise E.
    [Show full text]
  • Dryadlinq for Scientific Analyses
    DryadLINQ for Scientific Analyses Jaliya Ekanayake1,a, Atilla Soner Balkirc, Thilina Gunarathnea, Geoffrey Foxa,b, Christophe Poulaind, Nelson Araujod, Roger Bargad aSchool of Informatics and Computing, Indiana University Bloomington bPervasive Technology Institute, Indiana University Bloomington cDepartment of Computer Science, University of Chicago dMicrosoft Research {jekanaya, tgunarat, gcf}@indiana.edu, [email protected],{cpoulain,nelson,barga}@microsoft.com Abstract data-centered approach to parallel processing. In these frameworks, the data is staged in data/compute nodes Applying high level parallel runtimes to of clusters or large-scale data centers and the data/compute intensive applications is becoming computations are shipped to the data in order to increasingly common. The simplicity of the MapReduce perform data processing. HDFS allows Hadoop to programming model and the availability of open access data via a customized distributed storage system source MapReduce runtimes such as Hadoop, attract built on top of heterogeneous compute nodes, while more users around MapReduce programming model. DryadLINQ and CGL-MapReduce access data from Recently, Microsoft has released DryadLINQ for local disks and shared file systems. The simplicity of academic use, allowing users to experience a new these programming models enables better support for programming model and a runtime that is capable of quality of services such as fault tolerance and performing large scale data/compute intensive monitoring. analyses. In this paper, we present our experience in Although DryadLINQ comes with standard samples applying DryadLINQ for a series of scientific data such as Terasort, word count, its applicability for large analysis applications, identify their mapping to the scale data/compute intensive scientific applications is DryadLINQ programming model, and compare their not studied well.
    [Show full text]
  • Automatic and Explicit Parallelization Approaches for Mathematical Simulation Models
    Link¨opingStudies in Science and Technology Licentiate Thesis No. 1716 Automatic and Explicit Parallelization Approaches for Mathematical Simulation Models by Mahder Gebremedhin Department of Computer and Information Science Link¨opingUniversity SE-581 83 Link¨oping,Sweden Link¨oping2015 This is a Swedish Licentiate’s Thesis Swedish postgraduate education leads to a doctor’s degree and/or a licentiate’s degree. A doctor’s degree comprises 240 ECTS credits (4 year of full-time studies). A licentiate’s degree comprises 120 ECTS credits. Copyright c 2015 June Mahder Gebremedhin ISBN 978-91-7519-048-8 ISSN 0280–7971 Printed by LiU Tryck 2015 URL: http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-117338 Abstract The move from single-core processor systems to multi-core and many-processor systems comes with the requirement of implementing computations in a way that can utilize these multiple computational units efficiently. This task of writing efficient parallel algorithms will not be possible without improving programming languages and compilers to provide the supporting mecha- nisms. Computer aided mathematical modeling and simulation is one of the most computationally intensive areas of computer science. Even simplified models of physical systems can impose a considerable computational load on the processors at hand. Being able to take advantage of the potential computational power provided by multi-core systems is vital in this area of application. This thesis tries to address how to take advantage of the poten- tial computational power provided by these modern processors in order to improve the performance of simulations, especially for models in the Mod- elica modeling language compiled and simulated using the OpenModelica compiler and run-time environment.
    [Show full text]
  • Openmp a Parallel Programming Model for Shared Memory Architectures
    OpenMP A Parallel Programming Model for Shared Memory Architectures Paul Graham Edinburgh Parallel Computing Centre The University of Edinburgh March 1999 Version 1.1 Available from: http://www.epcc.ed.ac.uk/epcc-tec/documents/ Table of Contents 1 Introduction . 1 2 Shared memory platforms . 2 3 Why OpenMP? . 4 3.1 Background and Support for OpenMP . 5 3.2 A simple parallelisation example . 5 3.3 Special features of OpenMP . 6 4 The OpenMP Specification . 10 4.1 Parallelisation directives . 10 4.2 Parallel region construct . 10 4.3 Data environment constructs . 11 4.4 Work-sharing constructs . 15 4.5 Synchronisation constructs . 17 4.6 Conditional compilation . 19 5 Library Routines and Environment Variables . 20 5.1 Execution Environment Routines. 20 5.2 Lock Routines . 21 5.3 Environment Variables. 22 6 Performance and Scalability . 24 6.1 The Game of Life. 24 6.2 Performance. 26 7 References . 31 8 Acknowledgements . 33 A MPI version of the Game of Life . 35 B HPF version of the Game of Life . 39 Edinburgh Parallel Computing Centre 3 OpenMP 4 Technology Watch Report Introduction 1 Introduction Parallel programming on shared memory machines has always been an important area in high performance computing (HPC). However, the utilisation of such platforms has never been straightforward for the programmer. The Message Passing Interface (MPI) commonly used on massively parallel distributed memory architectures offers good scalability and port- ability, but is non-trivial to implement with codes originally written for serial machines. It also fails to take advantage of the architecture of shared memory platforms.
    [Show full text]