Université De Strasbourg École Doctorale Mathématiques
Total Page:16
File Type:pdf, Size:1020Kb
Université de Strasbourg École Doctorale Mathématiques, Sciences de l’Information et de l’Ingénieur Laboratoire des sciences de l’ingénieur, de l’informatique et de l’imagerie (ICube) THÈSE présentée par : Jean-François DOLLINGER Soutenue le : 01 juillet 2015 pour obtenir le grade de : Docteur de l’Université de Strasbourg discipline/spécialité : Informatique A framework for efficient execution on GPU and CPU+GPU systems Thèse dirigée par : M. CLAUSS Philippe Professeur, Unistra, Strasbourg Rapporteurs : M. COHEN Albert Directeur de Recherche INRIA, ENS, Paris M. BODIN François Professeur, IRISA, Rennes Examinateurs : Professeur, Universität des Saarlandes, Saar- M. HACK Sebastian brücken, Allemagne M. LOECHNER Vincent Maître de conférences, Unistra, Strasbourg Laboratoire des sciences de l’ingénieur, de l’informatiqueet de l’imagerie(ICube) UMR 7357 2 Contents 1 Résumé en Français 11 1.1 Introduction................................. 11 1.2 Prédiction et sélection adaptative de version de code . ........ 12 1.2.1 Introduction............................. 12 1.2.2 Vue d’ensemble du framework . 13 1.2.3 Equité ................................ 14 1.2.4 Prédiction .............................. 14 1.2.5 Expérimentations . .. .. 16 1.3 Calcul hétérogène : CPU vs GPU ..................... 17 1.3.1 Introduction............................. 17 1.3.2 CodeCPU.............................. 18 1.3.3 CodeGPU.............................. 18 1.3.4 Expérimentations . .. .. 19 1.4 Calculhétérogène:CPU+GPU . 19 1.4.1 Introduction............................. 19 1.4.2 Générationdecode ......................... 20 1.4.3 Répartitiondecharge. 21 1.4.4 Multiversioning ........................... 23 1.4.5 Consommationd’énergie . 23 1.4.6 Experimentations . .. .. 23 1.5 Parallélisationspéculative . ... 24 1.5.1 Introduction............................. 24 1.5.2 Problématique............................ 25 1.5.3 Machinevirtuelle .......................... 25 1.5.4 Contributions ............................ 25 2 Introduction 27 2.1 Context ................................... 27 2.2 Problematic ................................. 28 2.3 Outline.................................... 30 3 Context and related work 31 3.1 GenesisofGPUcomputing. 32 3.2 GPUarchitecture .............................. 34 3.2.1 CUDA................................ 36 3 4 CONTENTS 3.2.2 Processorspace ........................... 36 3.2.3 Softwaremodel ........................... 38 3.2.4 Memoryspace............................ 39 3.2.5 OpenCL............................... 41 3.2.6 CUDAvsOpenCL ......................... 42 3.3 Directivelanguages ............................. 42 3.4 Approaches to performance modelling . ... 43 3.5 Optimizationtechniques . 44 3.5.1 Hand-tuning vs. Automatic optimization . .. 44 3.5.2 Staticmethods ........................... 45 3.5.3 Hybridmethods........................... 46 3.5.4 Dynamicmethods.......................... 48 3.5.5 Conclusion.............................. 48 3.6 Heterogeneouscomputing . 49 3.6.1 Staticpartitioning . 50 3.6.2 Hybridpartitioning. 51 3.6.3 Dynamicpartitioning. 51 3.6.4 Conclusion.............................. 52 3.7 PolytopeModel............................... 53 3.7.1 SCoP................................. 54 3.7.2 Polyhedral model illustration . 56 3.7.3 Accessfunctions........................... 56 3.7.4 Scatteringmatrix . .. .. 58 3.7.5 Dependence ............................. 61 3.7.6 Transformations. .. .. 62 3.7.7 Integerpointscounting. 63 3.8 CLooG.................................... 64 3.9 PLUTO ................................... 64 3.10Par4All ................................... 65 3.11R-Stream .................................. 65 3.12PPCG .................................... 65 3.13Skeletoncompilers ............................. 66 3.14Libraries................................... 66 3.15 Speculativeparallelization . .... 67 4 Code versioning and profiling for GPU 69 4.1 Introduction................................. 69 4.2 RelatedWork ................................ 71 4.3 Frameworkoverview ............................ 73 4.4 Profiling................................... 75 4.4.1 Equity................................ 75 4.4.2 Staticperformancefactors . 76 4.4.3 Dynamicperformancefactors . 76 4.4.4 CUDAgridsizeevaluation . 86 4.5 Runtimeprediction ............................. 88 CONTENTS 5 4.5.1 Prediction .............................. 88 4.6 Experiments................................. 90 4.6.1 Testbed ............................... 90 4.6.2 Predictionaccuracy. 93 4.7 Perspectives and conclusion . 94 5 Heterogeneous computing 95 5.1 CPU vs GPU execution: a dynamic approach . 96 5.1.1 Introduction............................. 96 5.1.2 CPUvsGPUattempt ....................... 97 5.1.3 EarlyterminationofCPUandGPUcodes . 99 5.1.4 Experimentations . 102 5.1.5 Perspectiveandlimitations. 105 5.2 CPU vs GPU execution: a hybrid approach . 106 5.3 CPU+GPUjointexecution. 108 5.3.1 Introduction............................. 108 5.3.2 Codegeneration. .. .. 111 5.3.3 CPU+GPURuntime . 113 5.3.4 Evaluation.............................. 117 5.3.5 Multiversioning . 120 5.3.6 Power-guided scheduling . 131 5.3.7 Perspectives and conclusion . 134 6 Thread Level Speculation 139 6.1 CPUspeculativeexecution . 139 6.1.1 Introduction............................. 139 6.2 Overviewofoursystem. .. .. 141 6.3 Binaryskeletons............................... 144 6.3.1 Guardingcode ........................... 146 6.3.2 Initializationcode. 146 6.3.3 Verification code for speculative parallelization . ........ 147 6.4 Memorybackup............................... 148 6.5 Experimentalresults ............................ 150 6.6 Memorybackupextensions. 155 6.7 GPUextensionproposal . .. .. 155 6.7.1 Verificationcode . .. .. 155 6.7.2 Memoryaddressspace . 156 6.7.3 Finding adequate block size . 157 6.7.4 CPU+GPUexecution. 157 6.8 Conclusions and perspectives . 158 7 Conclusion 161 7.1 Contributions ................................ 161 7.2 Perspectives ................................. 163 Bibliography 165 6 CONTENTS List of Figures 1.2 Comparaison entre temps d’exécution prédit (barre gauche) et effectif (barre droite) sur PolyBench avec le jeu de données standard sur GTX 590. ..................................... 17 1.3 Comparaison entre temps d’exécution prédit (barre gauche) et effectif (barre droite) sur PolyBench avec le jeu de données standard sur GTX 680. ..................................... 17 1.4 Temps d’exécution normalisé pour le jeu de données standard sur Poly- Bench .................................... 20 1.5 Speedup to execution time on CPU or GPU alone. .. 24 3.1 Recomputed theoretical peak performance trends between Intel CPUs andNvidiaGPUs. ............................. 32 3.2 Nvidia discrete graphics card architecture. ....... 35 3.3 Insight of a Streaming Multiprocessor (SM) organization......... 37 3.4 CUDAexecutionmodel. .......................... 38 3.5 Illustration of a loop nest and its associated polyhedron......... 56 3.6 Constraint inequalities and associated constraint matrix......... 56 4.1 Comparison of actual execution time per iteration for several block - tile sizes for gemm (matrix size = 1024 1024). ............... 70 ∗ 4.2 Example of a CPU ranking table and prediction for gemm........ 72 4.3 Global framework workflow overview. .. 73 4.4 Comparison (top) and deviation (bottom) of actual execution time per iteration to empirically demonstrate commutativity of the block dimen- sions for gemm, Block/Tile size: 32 16/32 16, NK = 2000...... 75 × × 4.5 Comparison between profiled and measured execution times per iteration (1)....................................... 79 4.6 Comparison between profiled and effective execution times per iteration (2)....................................... 80 4.7 Sequential parameters influence for different tile configurations for syrk. 81 4.8 Bandwidth evaluation on Asus P8P67-Pro motherboard with Asus GTX 590 GPU and Asus P8P67-Deluxe with Nvidia GTX 680 GPU. 82 4.9 Profilingflowchart.............................. 83 7 8 LIST OF FIGURES 4.10 Demonstration of the capability of our profiler to map the actual execu- tion time per iteration measurements (solid) for gemm, Block/Tile size: 16 16/128 128, NK = 2000. The profiler makes use of a paramet- × × ric sliding window to perform measurements in a wider spectrum (solid, cross). This configuration is compared to a single point based decision, with a maximum allowed deviation of 10% (dashed, square) and 5% (dashed,circle). ............................... 85 4.11 A code version (a) and its associated prediction code (b)......... 89 4.12 Comparison between prediction (left bar) and real (right bar) execution times on PolyBench with small dataset on GTX590. 90 4.13 Comparison between prediction (left bar) and real (right bar) execution times on PolyBench with standard dataset on GTX590. .. 91 4.14 Comparison between prediction (left bar) and real (right bar) execution times on PolyBench with large dataset on GTX590. 91 4.15 Comparison between prediction (left bar) and real (right bar) execution times on PolyBench with small dataset on GTX680. 92 4.16 Comparison between prediction (left bar) and real (right bar) execution times on PolyBench with standard dataset on GTX680. .. 92 4.17 Comparison between prediction (left bar) and real (right bar) execution times on PolyBench with large dataset on GTX680. 92 5.1 GPU kernel termination algorithm with thread election. ......... 97 5.2 Comparison of execution time between zero-copy (accessing central me- mory) and