DOCSLIB.ORG

Cс 2006 by Gang Ren. All Rights Reserved

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

c 2006 by Gang Ren. All rights reserved. COMPILING VECTOR PROGRAMS FOR SIMD DEVICES BY GANG REN B.E., Zhejiang University, 1998 M.S., Chinese Academy of Sciences, 2001 DISSERTATION Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate College of the University of Illinois at Urbana-Champaign, 2006 Urbana, Illinois Abstract As an effective way of utilizing data parallelism in applications, SIMD architecture has been adopted by most today’s microprocessors. Using intrinsic functions and automatic compilation are common programming methods for today’s SIMD devices. However, neither methods can provide enough programmability and performance at the same time. Many issues must be addressed to generate efficient SIMD code. For example, most SIMD devices only support memory accesses on contiguous and aligned sections. Additional permutation instructions are needed for non-contiguous and/or misaligned references. Such overhead can cancel all performance benefits from SIMD computation. VINCI, or Vector I-code Novel Compilation Infrastructure, is proposed in this thesis. VINCI focuses on translating vector programs into efficient code for SIMD devices. Vectors in input programs can have arbitrary length, strides, and alignment settings. However, vectors required by SIMD devices must have the same fixed length, unit strides, and aligned addresses. VINCI employs a sequence of program transformations to convert all vectors into such specific format. VINCI also includes several optimization algorithms. The optimization algorithm on data permutations is of great importance. By unifying all forms of data permutations into the explicit representation, the optimization algorithm can reduce the number of data permuta- tions in vector programs by propagating them across statements and merging them whenever possible. In addition, an efficient code generation algorithm is included to generate native permutation instructions from vector permutation operations. Besides, any common compiler analysis and optimizations were also extended for vector iii representation and included in VINCI. Two examples are def-use analysis and copy prop- agation. In addition, two domain-specific optimization techniques for DSP programs are also extended for vector programs. These optimizations are necessary to delivery the final performance on SIMD devices. VINCI was implemented on the HiLO compiler, an internal compiler used in SPIRAL. Experiments were conducted on two platforms, VMX and SSE2. Testing applications include both automatically-generated programs and manually-written kernels. The results show that up to 77% of the permutation instructions are eliminated and, as a result, the average performance improvement is 48% on VMX and 68% on SSE2. For several applications, near perfect speedups have been achieved on both platforms. iv To My Family v Acknowledgements First, I would like to express my deep and sincere gratitude to my advisor, Professor David Padua, for guiding and supporting me throughout the whole Ph.D. study. His advices, insights, and encouragement helped me in all the time of this five-year research. I benefited tremendously from his wide knowledge and logical way of thinking. The completion of this thesis would not have been possible without his guidance. I would like to sincerely thank Dr. Peng Wu for her detailed and constructive suggestions. From the beginning of my graduate study, I have learned a lot from all the interesting and insightful discussions with her. It was such a great experience working with her as an intern at IBM. I owe my warm gratitude to the members of my thesis committee, Professor Vikram Adve, Professor Williams Harrison and Professor Wen-Mei Hwu. Their insightful advices are invaluable for my thesis research. I am indebted to them for reading my dissertation and providing penetrating feedback. I am also indebted to Dr. Aart Bik and Dr. Chris Lattner for donating their time to read draft versions of the thesis and giving me helpful comments. I would like to thank Professor Maria Garzaran for her comments on my early work on automatic performance tuning and for her selfless help on my thesis research. I would like to extend my thanks to Professor Josep Torrellas for his interesting classes and for his extremely helpful recommendation letter. I want to thank Dr. Alexandre Eichenberger and the other members of the Cell compiler group at IBM Research. It was a wonderful experience working with these smart and friendly vi people. My thank also goes to all current and previous members of Polaris group, especially Jianxin Xiong, Jiajing Zhu, Zehra Sura, David Wong, Xiaoming Li and Nick Rizzolo. I am also grateful to Sheila Clark for her great administrative support during my graduate study. I owe my loving thanks to my parents and grandparents. Without their encouragement it would have been absolutely impossible for me to finish my thesis work. My special thank goes to my grandfather, Dingyuan Chen, for the 30-year endless love and support. Last but notleast,mywarmestthankisduetomywife, Shengnan, for making my life complete. vii Table of Contents ListofFigures....................................... xi Chapter1 Introduction.................................. 1 Chapter2 OverviewofSIMDDevices.......................... 8 2.1ArchitectureofSIMDDevices.......................... 8 2.2CharacteristicsofToday’sSIMDDevices.................... 11 2.3PermutationInstructionSet........................... 13 2.4ProgrammingSIMDDevices........................... 16 Chapter 3 Compiling for SIMD Devices ......................... 20 3.1AnOverviewofSIMDCompilation....................... 20 3.2ExperimentswithSIMDCompilers....................... 22 3.2.1 BerkeleyMultimediaWorkload..................... 22 3.2.2 ExperimentalResults........................... 23 3.3VINCI:VectorI-codeNovelCompilationInfrastructure............ 25 Chapter4 VectorandVectorOperations........................ 26 4.1VectorRepresentation.............................. 26 4.1.1 VectorandVectorExpression...................... 26 4.1.2 VectorOperations............................. 29 4.2DataPermutationonVectors.......................... 30 4.2.1 PermutationOperation.......................... 30 4.2.2 PropertiesofthePermutationOperation................ 32 4.2.3 ReductionandReplication........................ 34 4.2.4 PermutationOperationsinVectorI-codePrograms.......... 35 Chapter5 NormalizingVectorPrograms........................ 37 5.1NormalizingVectorswithNon-unitStrides................... 37 5.1.1 Stridedload................................ 38 5.1.2 Stridedstore............................... 39 5.2CoalescingPartiallyUtilizedPermutations................... 40 5.3 Normalizing Vectors cross Loop Boundaries .................. 41 5.4NormalizingVectorswithMisalignment..................... 42 5.5OtherSourcesofDataPermutations...................... 43 viii Chapter 6 Loop Unrolling and Vector Coalescing .................... 45 6.1ArrayExpansion................................. 46 6.2CoalescingVectorStatements.......................... 50 6.3 Unrolling Loops with Data Permutations .................... 51 Chapter7 DataPermutationOptimization....................... 53 7.1AnOverviewoftheOptimizationAlgorithm.................. 53 7.2 Propagation along Def-use Chains . ..................... 55 7.2.1 ReshapingPermutations......................... 57 7.2.2 PermutationDecomposition....................... 58 7.3 Propagation within a Statement ......................... 63 7.4OptimalityAnalysis................................ 65 7.4.1 NP-CompletenessoftheProblem.................... 65 7.4.2 HeuristicSolutions............................ 69 7.5TheGlobalOptimizationAlgorithm....................... 70 7.5.1 ExtensiononIf-Statements........................ 70 7.5.2 ExtensiononLoopswithoutBackwardDependences......... 72 7.5.3 ExtensiononLoopswithBackwardDependences........... 74 7.6OptimizingSpecialPermutations........................ 74 7.7RelatedWorkonDataPermutationOptimization............... 75 Chapter8 SIMDCodeGeneration............................ 77 8.1TheCodeGenerationAlgorithm......................... 77 8.2GeneratingSIMDPermutationInstructions.................. 78 8.2.1 Translating Permute into vperm Instructions............. 79 8.2.2 Mapping vperm toNativeInstructions................. 83 8.2.3 GeneratingReductionandReplication................. 83 Chapter9 Mixed-ModeSIMDCompilation....................... 85 9.1AMotivatingExample.............................. 85 9.2Mixed-ModeExtensiononVINCI........................ 86 9.3Discussion..................................... 88 Chapter10OtherCompilerRoutinesinVINCI.................... 90 10.1Def-useAnalysisonVectorProgram...................... 90 10.2 Vector Copy Propagation ............................. 91 Chapter11ImplementationontheHiLOCompiler.................. 94 11.1TheSPLCompiler................................ 94 11.2TheI-CodeLanguage............................... 96 11.3TheHiLOCompiler............................... 96 11.4ImplementationonHiLO............................. 97 Chapter12DomainSpecificOptimizationforDSPPrograms............. 100 12.1SimplificationofArithmeticOperations..................... 100 12.2InstructionClustering.............................. 101 ix Chapter13ExperimentalResults............................ 104 13.1TheSPIRALSystem............................... 104 13.2ExperimentSetups................................ 105 13.3StaticEvaluation................................
Recommended publications
  • 07 Vectorization for Intel C++ & Fortran Compiler .Pdf

    07 Vectorization for Intel C++ & Fortran Compiler .Pdf

    Vectorization for Intel® C++ & Fortran Compiler Presenter: Georg Zitzlsberger Date: 10-07-2015 1 Agenda • Introduction to SIMD for Intel® Architecture • Compiler & Vectorization • Validating Vectorization Success • Reasons for Vectorization Fails • Intel® Cilk™ Plus • Summary 2 Optimization Notice Copyright © 2015, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. Vectorization • Single Instruction Multiple Data (SIMD): . Processing vector with a single operation . Provides data level parallelism (DLP) . Because of DLP more efficient than scalar processing • Vector: . Consists of more than one element . Elements are of same scalar data types (e.g. floats, integers, …) • Vector length (VL): Elements of the vector A B AAi i BBi i A B Ai i Bi i Scalar Vector Processing + Processing + C CCi i C Ci i VL 3 Optimization Notice Copyright © 2015, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. Evolution of SIMD for Intel Processors Present & Future: Goal: Intel® MIC Architecture, 8x peak FLOPs (FMA) over 4 generations! Intel® AVX-512: • 512 bit Vectors • 2x FP/Load/FMA 4th Generation Intel® Core™ Processors Intel® AVX2 (256 bit): • 2x FMA peak Performance/Core • Gather Instructions 2nd Generation 3rd Generation Intel® Core™ Processors Intel® Core™ Processors Intel® AVX (256 bit): • Half-float support • 2x FP Throughput • Random Numbers • 2x Load Throughput Since 1999: Now & 2010 2012 2013 128 bit Vectors Future Time 4 Optimization Notice
  • Analysis of SIMD Applicability to SHA Algorithms O

    Analysis of SIMD Applicability to SHA Algorithms O

    1 Analysis of SIMD Applicability to SHA Algorithms O. Aciicmez Abstract— It is possible to increase the speed and throughput of The remainder of the paper is organized as follows: In an algorithm using parallelization techniques. Single-Instruction section 2 and 3, we introduce SIMD concept and the SIMD Multiple-Data (SIMD) is a parallel computation model, which has architecture of Intel including MMX technology and SSE already employed by most of the current processor families. In this paper we will analyze four SHA algorithms and determine extensions. Section 4 describes SHA algorithm and Section possible performance gains that can be achieved using SIMD 5 discusses the possible improvements on SHA performance parallelism. We will point out the appropriate parts of each that can be achieved by using SIMD instructions. algorithm, where SIMD instructions can be used. II. SIMD PARALLEL PROCESSING I. INTRODUCTION Single-instruction multiple-data execution model allows Today the security of a cryptographic mechanism is not the several data elements to be processed at the same time. The only concern of cryptographers. The heavy communication conventional scalar execution model, which is called single- traffic on contemporary very large network of interconnected instruction single-data (SISD) deals only with one pair of data devices demands a great bandwidth for security protocols, and at a time. The programs using SIMD instructions can run much hence increasing the importance of speed and throughput of a faster than their scalar counterparts. However SIMD enabled cryptographic mechanism. programs are harder to design and implement. A straightforward approach to improve cryptographic per- The most common use of SIMD instructions is to perform formance is to implement cryptographic algorithms in hard- parallel arithmetic or logical operations on multiple data ware.
  • Deterministic Execution of Multithreaded Applications

    Deterministic Execution of Multithreaded Applications

    DETERMINISTIC EXECUTION OF MULTITHREADED APPLICATIONS FOR RELIABILITY OF MULTICORE SYSTEMS DETERMINISTIC EXECUTION OF MULTITHREADED APPLICATIONS FOR RELIABILITY OF MULTICORE SYSTEMS Proefschrift ter verkrijging van de graad van doctor aan de Technische Universiteit Delft, op gezag van de Rector Magnificus prof. ir. K. C. A. M. Luyben, voorzitter van het College voor Promoties, in het openbaar te verdedigen op vrijdag 19 juni 2015 om 10:00 uur door Hamid MUSHTAQ Master of Science in Embedded Systems geboren te Karachi, Pakistan Dit proefschrift is goedgekeurd door de promotor: Prof. dr. K. L. M. Bertels Copromotor: Dr. Z. Al-Ars Samenstelling promotiecommissie: Rector Magnificus, voorzitter Prof. dr. K. L. M. Bertels, Technische Universiteit Delft, promotor Dr. Z. Al-Ars, Technische Universiteit Delft, copromotor Independent members: Prof. dr. H. Sips, Technische Universiteit Delft Prof. dr. N. H. G. Baken, Technische Universiteit Delft Prof. dr. T. Basten, Technische Universiteit Eindhoven, Netherlands Prof. dr. L. J. M Rothkrantz, Netherlands Defence Academy Prof. dr. D. N. Pnevmatikatos, Technical University of Crete, Greece Keywords: Multicore, Fault Tolerance, Reliability, Deterministic Execution, WCET The work in this thesis was supported by Artemis through the SMECY project (grant 100230). Cover image: An Artist’s impression of a view of Saturn from its moon Titan. The image was taken from http://www.istockphoto.com and used with permission. ISBN 978-94-6186-487-1 Copyright © 2015 by H. Mushtaq All rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means without the prior written permission of the copyright owner.
  • SIMD Extensions

    SIMD Extensions

    SIMD Extensions PDF generated using the open source mwlib toolkit. See http://code.pediapress.com/ for more information. PDF generated at: Sat, 12 May 2012 17:14:46 UTC Contents Articles SIMD 1 MMX (instruction set) 6 3DNow! 8 Streaming SIMD Extensions 12 SSE2 16 SSE3 18 SSSE3 20 SSE4 22 SSE5 26 Advanced Vector Extensions 28 CVT16 instruction set 31 XOP instruction set 31 References Article Sources and Contributors 33 Image Sources, Licenses and Contributors 34 Article Licenses License 35 SIMD 1 SIMD Single instruction Multiple instruction Single data SISD MISD Multiple data SIMD MIMD Single instruction, multiple data (SIMD), is a class of parallel computers in Flynn's taxonomy. It describes computers with multiple processing elements that perform the same operation on multiple data simultaneously. Thus, such machines exploit data level parallelism. History The first use of SIMD instructions was in vector supercomputers of the early 1970s such as the CDC Star-100 and the Texas Instruments ASC, which could operate on a vector of data with a single instruction. Vector processing was especially popularized by Cray in the 1970s and 1980s. Vector-processing architectures are now considered separate from SIMD machines, based on the fact that vector machines processed the vectors one word at a time through pipelined processors (though still based on a single instruction), whereas modern SIMD machines process all elements of the vector simultaneously.[1] The first era of modern SIMD machines was characterized by massively parallel processing-style supercomputers such as the Thinking Machines CM-1 and CM-2. These machines had many limited-functionality processors that would work in parallel.
  • CUDA Toolkit 4.2 CURAND Guide

    CUDA Toolkit 4.2 CURAND Guide

    CUDA Toolkit 4.2 CURAND Guide PG-05328-041_v01 | March 2012 Published by NVIDIA Corporation 2701 San Tomas Expressway Santa Clara, CA 95050 Notice ALL NVIDIA DESIGN SPECIFICATIONS, REFERENCE BOARDS, FILES, DRAWINGS, DIAGNOSTICS, LISTS, AND OTHER DOCUMENTS (TOGETHER AND SEPARATELY, "MATERIALS") ARE BEING PROVIDED "AS IS". NVIDIA MAKES NO WARRANTIES, EXPRESSED, IMPLIED, STATUTORY, OR OTHERWISE WITH RESPECT TO THE MATERIALS, AND EXPRESSLY DISCLAIMS ALL IMPLIED WARRANTIES OF NONINFRINGEMENT, MERCHANTABILITY, AND FITNESS FOR A PARTICULAR PURPOSE. Information furnished is believed to be accurate and reliable. However, NVIDIA Corporation assumes no responsibility for the consequences of use of such information or for any infringement of patents or other rights of third parties that may result from its use. No license is granted by implication or otherwise under any patent or patent rights of NVIDIA Corporation. Specifications mentioned in this publication are subject to change without notice. This publication supersedes and replaces all information previously supplied. NVIDIA Corporation products are not authorized for use as critical components in life support devices or systems without express written approval of NVIDIA Corporation. Trademarks NVIDIA, CUDA, and the NVIDIA logo are trademarks or registered trademarks of NVIDIA Corporation in the United States and other countries. Other company and product names may be trademarks of the respective companies with which they are associated. Copyright Copyright ©2005-2012 by NVIDIA Corporation. All rights reserved. CUDA Toolkit 4.2 CURAND Guide PG-05328-041_v01 | 1 Portions of the MTGP32 (Mersenne Twister for GPU) library routines are subject to the following copyright: Copyright ©2009, 2010 Mutsuo Saito, Makoto Matsumoto and Hiroshima University.
  • Unit: 4 Processes and Threads in Distributed Systems

    Unit: 4 Processes and Threads in Distributed Systems

    Unit: 4 Processes and Threads in Distributed Systems Thread A program has one or more locus of execution. Each execution is called a thread of execution. In traditional operating systems, each process has an address space and a single thread of execution. It is the smallest unit of processing that can be scheduled by an operating system. A thread is a single sequence stream within in a process. Because threads have some of the properties of processes, they are sometimes called lightweight processes. In a process, threads allow multiple executions of streams. Thread Structure Process is used to group resources together and threads are the entities scheduled for execution on the CPU. The thread has a program counter that keeps track of which instruction to execute next. It has registers, which holds its current working variables. It has a stack, which contains the execution history, with one frame for each procedure called but not yet returned from. Although a thread must execute in some process, the thread and its process are different concepts and can be treated separately. What threads add to the process model is to allow multiple executions to take place in the same process environment, to a large degree independent of one another. Having multiple threads running in parallel in one process is similar to having multiple processes running in parallel in one computer. Figure: (a) Three processes each with one thread. (b) One process with three threads. In former case, the threads share an address space, open files, and other resources. In the latter case, process share physical memory, disks, printers and other resources.
  • A Short History of Computational Complexity

    A Short History of Computational Complexity

    The Computational Complexity Column by Lance FORTNOW NEC Laboratories America 4 Independence Way, Princeton, NJ 08540, USA [email protected] http://www.neci.nj.nec.com/homepages/fortnow/beatcs Every third year the Conference on Computational Complexity is held in Europe and this summer the University of Aarhus (Denmark) will host the meeting July 7-10. More details at the conference web page http://www.computationalcomplexity.org This month we present a historical view of computational complexity written by Steve Homer and myself. This is a preliminary version of a chapter to be included in an upcoming North-Holland Handbook of the History of Mathematical Logic edited by Dirk van Dalen, John Dawson and Aki Kanamori. A Short History of Computational Complexity Lance Fortnow1 Steve Homer2 NEC Research Institute Computer Science Department 4 Independence Way Boston University Princeton, NJ 08540 111 Cummington Street Boston, MA 02215 1 Introduction It all started with a machine. In 1936, Turing developed his theoretical com- putational model. He based his model on how he perceived mathematicians think. As digital computers were developed in the 40's and 50's, the Turing machine proved itself as the right theoretical model for computation. Quickly though we discovered that the basic Turing machine model fails to account for the amount of time or memory needed by a computer, a critical issue today but even more so in those early days of computing. The key idea to measure time and space as a function of the length of the input came in the early 1960's by Hartmanis and Stearns.
  • Extensions to Instruction Sets

    Extensions to Instruction Sets

    Extensions to Instruction Sets Ricardo Manuel Meira Ferrão Luis Departamento de Informática, Universidade do Minho 4710 - 057 Braga, Portugal [email protected] Abstract. This communication analyses extensions to instruction sets in general purpose processors, together with their evolution and significant innovation. It begins with an overview of specific multimedia and digital signal processing requirements and then it focuses on feature detection to properly identify the adequate extension. A brief performance evaluation on two competitive processors closes the communication. 1 Introduction The purpose of this communication is to analyse extensions to instruction sets, describe the specific multimedia and digital signal processing (DSP) requirements of a processor, detail how to detect these extensions and verify if the processor supports them. To achieve this functionality an IA32 architecture example code will be demonstrated. To study the instruction sets supported by each processor, an analysis is performed to compare and distinguish each technology in terms of new extensions to instruction sets and architecture evolution, mapping a table that lists the instruction sets supported by each processor. To clearly identify these differences between architectures, tests are performed with an application example, determining comparable results of two competitive processors. In this communication, Section 2 gives an overview of multimedia and digital signal processing requirements, Section 3 explains the feature detection of these instructions, Section 4 details some of the extensions supported by IA32 based processors, Section 5 tests and describes the performance of each technology and analyses the results. Section 6 concludes this communication. 2 Multimedia and DSP Requirements Computer industries needed a way to improve their multimedia capabilities because of the demand for 3D graphics, video and other multimedia applications such as games.
  • Deterministic Parallel Fixpoint Computation

    Deterministic Parallel Fixpoint Computation

    Deterministic Parallel Fixpoint Computation SUNG KOOK KIM, University of California, Davis, U.S.A. ARNAUD J. VENET, Facebook, Inc., U.S.A. ADITYA V. THAKUR, University of California, Davis, U.S.A. Abstract interpretation is a general framework for expressing static program analyses. It reduces the problem of extracting properties of a program to computing an approximation of the least fixpoint of a system of equations. The de facto approach for computing this approximation uses a sequential algorithm based on weak topological order (WTO). This paper presents a deterministic parallel algorithm for fixpoint computation by introducing the notion of weak partial order (WPO). We present an algorithm for constructing a WPO in almost-linear time. Finally, we describe Pikos, our deterministic parallel abstract interpreter, which extends the sequential abstract interpreter IKOS. We evaluate the performance and scalability of Pikos on a suite of 1017 C programs. When using 4 cores, Pikos achieves an average speedup of 2.06x over IKOS, with a maximum speedup of 3.63x. When using 16 cores, Pikos achieves a maximum speedup of 10.97x. CCS Concepts: • Software and its engineering → Automated static analysis; • Theory of computation → Program analysis. Additional Key Words and Phrases: Abstract interpretation, Program analysis, Concurrency ACM Reference Format: Sung Kook Kim, Arnaud J. Venet, and Aditya V. Thakur. 2020. Deterministic Parallel Fixpoint Computation. Proc. ACM Program. Lang. 4, POPL, Article 14 (January 2020), 33 pages. https://doi.org/10.1145/3371082 1 INTRODUCTION Program analysis is a widely adopted approach for automatically extracting properties of the dynamic behavior of programs [Balakrishnan et al.
  • Intel® Software Guard Extensions (Intel® SGX)

    Intel® Software Guard Extensions (Intel® SGX)

    Intel® Software Guard Extensions (Intel® SGX) Developer Guide Intel(R) Software Guard Extensions Developer Guide Legal Information No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Intel disclaims all express and implied warranties, including without limitation, the implied war- ranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade. This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel rep- resentative to obtain the latest forecast, schedule, specifications and roadmaps. The products and services described may contain defects or errors known as errata which may cause deviations from published specifications. Current characterized errata are available on request. Intel technologies features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at Intel.com, or from the OEM or retailer. Copies of documents which have an order number and are referenced in this document may be obtained by calling 1-800-548-4725 or by visiting www.intel.com/design/literature.htm. Intel, the Intel logo, Xeon, and Xeon Phi are trademarks of Intel Corporation in the U.S. and/or other countries. Optimization Notice Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations.
  • Byzantine Fault Tolerance for Nondeterministic Applications Bo Chen

    Byzantine Fault Tolerance for Nondeterministic Applications Bo Chen

    BYZANTINE FAULT TOLERANCE FOR NONDETERMINISTIC APPLICATIONS BO CHEN Bachelor of Science in Computer Engineering Nanjing University of Science & Technology, China July, 2006 Submitted in partial fulfillment of the requirements for the degree MASTER OF SCIENCE IN COMPUTER ENGINEERING at the CLEVELAND STATE UNIVERSITY December, 2008 The thesis has been approved for the Department of ELECTRICAL AND COMPUTER ENGINEERING and the College of Graduate Studies by ______________________________________________ Thesis Committee Chairperson, Dr. Wenbing Zhao ____________________ Department/Date _____________________________________________ Dr. Yongjian Fu ______________________ Department/Date _____________________________________________ Dr. Ye Zhu ______________________ Department/Date ACKNOWLEDGEMENT First, I must thank my thesis supervisor, Dr. Wenbing Zhao, for his patience, careful thought, insightful commence and constant support during my two years graduate study and my career. I feel very fortunate for having had the chance to work closely with him and this thesis is as much a product of his guidance as it is of my effort. The other member of my thesis committee, Dr. Yongjian Fu and Dr. Ye Zhu suggested many important improvements to this thesis and interesting directions for future work. I greatly appreciate their suggestions. It has been a pleasure to be a graduate student in Secure and Dependable System Group. I want to thank all the group members: Honglei Zhang and Hua Chai for the discussion we had. I also wish to thank Dr. Fuqin Xiong for his help in advising the details of the thesis submission process. I am grateful to my parents for their support and understanding over the years, especially in the month leading up to this thesis. Above all, I want to thank all my friends who made my life great when I was preparing and writing this thesis.
  • Internally Deterministic Parallel Algorithms Can Be Fast

    Internally Deterministic Parallel Algorithms Can Be Fast

    Internally Deterministic Parallel Algorithms Can Be Fast Guy E. Blelloch∗ Jeremy T. Finemany Phillip B. Gibbonsz Julian Shun∗ ∗Carnegie Mellon University yGeorgetown University zIntel Labs, Pittsburgh [email protected] jfi[email protected] [email protected] [email protected] Abstract 1. Introduction The virtues of deterministic parallelism have been argued for One of the key challenges of parallel programming is dealing with decades and many forms of deterministic parallelism have been nondeterminism. For many computational problems, there is no in- described and analyzed. Here we are concerned with one of the herent nondeterminism in the problem statement, and indeed a se- strongest forms, requiring that for any input there is a unique de- rial program would be deterministic—the nondeterminism arises pendence graph representing a trace of the computation annotated solely due to the parallel program and/or due to the parallel ma- with every operation and value. This has been referred to as internal chine and its runtime environment. The challenges of nondetermin- determinism, and implies a sequential semantics—i.e., considering ism have been recognized and studied for decades [23, 24, 37, 42]. any sequential traversal of the dependence graph is sufficient for Steele’s 1990 paper, for example, seeks “to prevent the behavior analyzing the correctness of the code. In addition to returning deter- of the program from depending on any accidents of execution or- ministic results, internal determinism has many advantages includ- der that can arise from the indeterminacy” of asynchronous pro- ing ease of reasoning about the code, ease of verifying correctness, grams [42].