OpenMP Application Programming Interface Version 5.1 November 2020 Copyright c 1997-2020 OpenMP Architecture Review Board. Permission to copy without fee all or part of this material is granted, provided the OpenMP Architecture Review Board copyright notice and the title of this document appear. Notice is given that copying is by permission of the OpenMP Architecture Review Board. This page intentionally left blank in published version. Contents 1 Overview of the OpenMP API1 1.1 Scope . .1 1.2 Glossary . .2 1.2.1 Threading Concepts . .2 1.2.2 OpenMP Language Terminology . .2 1.2.3 Loop Terminology . .9 1.2.4 Synchronization Terminology . 10 1.2.5 Tasking Terminology . 12 1.2.6 Data Terminology . 14 1.2.7 Implementation Terminology . 18 1.2.8 Tool Terminology . 19 1.3 Execution Model . 22 1.4 Memory Model . 25 1.4.1 Structure of the OpenMP Memory Model . 25 1.4.2 Device Data Environments . 26 1.4.3 Memory Management . 27 1.4.4 The Flush Operation . 27 1.4.5 Flush Synchronization and Happens Before .................. 29 1.4.6 OpenMP Memory Consistency . 30 1.5 Tool Interfaces . 31 1.5.1 OMPT . 32 1.5.2 OMPD . 32 1.6 OpenMP Compliance . 33 1.7 Normative References . 33 1.8 Organization of this Document . 35 i 2 Directives 37 2.1 Directive Format . 38 2.1.1 Fixed Source Form Directives . 43 2.1.2 Free Source Form Directives . 44 2.1.3 Stand-Alone Directives . 45 2.1.4 Array Shaping . 45 2.1.5 Array Sections . 46 2.1.6 Iterators . 49 2.2 Conditional Compilation . 52 2.2.1 Fixed Source Form Conditional Compilation Sentinels . 52 2.2.2 Free Source Form Conditional Compilation Sentinel . 53 2.3 Variant Directives . 53 2.3.1 OpenMP Context . 53 2.3.2 Context Selectors . 55 2.3.3 Matching and Scoring Context Selectors . 59 2.3.4 Metadirectives . 60 2.3.5 Declare Variant Directive . 63 2.3.6 dispatch Construct . 69 2.4 Internal Control Variables . 71 2.4.1 ICV Descriptions . 72 2.4.2 ICV Initialization . 74 2.4.3 Modifying and Retrieving ICV Values . 77 2.4.4 How ICVs are Scoped . 79 2.4.4.1 How the Per-Data Environment ICVs Work . 81 2.4.5 ICV Override Relationships . 82 2.5 Informational and Utility Directives . 83 2.5.1 requires Directive . 83 2.5.2 Assume Directive . 86 2.5.3 nothing Directive . 89 2.5.4 error Directive . 90 2.6 parallel Construct . 92 2.6.1 Determining the Number of Threads for a parallel Region . 96 2.6.2 Controlling OpenMP Thread Affinity . 98 ii OpenMP API – Version 5.1 November 2020 2.7 teams Construct . 100 2.8 masked Construct . 104 2.9 scope Construct . 106 2.10 Worksharing Constructs . 108 2.10.1 sections Construct . 109 2.10.2 single Construct . 112 2.10.3 workshare Construct . 114 2.11 Loop-Related Directives . 117 2.11.1 Canonical Loop Nest Form . 117 2.11.2 Consistent Loop Schedules . 125 2.11.3 order Clause . 125 2.11.4 Worksharing-Loop Construct . 126 2.11.4.1 Determining the Schedule of a Worksharing-Loop . 133 2.11.5 SIMD Directives . 134 2.11.5.1 simd Construct . 134 2.11.5.2 Worksharing-Loop SIMD Construct . 138 2.11.5.3 declare simd Directive . 140 2.11.6 distribute Loop Constructs . 143 2.11.6.1 distribute Construct . 143 2.11.6.2 distribute simd Construct . 147 2.11.6.3 Distribute Parallel Worksharing-Loop Construct . 148 2.11.6.4 Distribute Parallel Worksharing-Loop SIMD Construct . 149 2.11.7 loop Construct . 151 2.11.8 scan Directive . 154 2.11.9 Loop Transformation Constructs . 157 2.11.9.1 tile Construct . 158 2.11.9.2 unroll Construct . 160 2.12 Tasking Constructs . 161 2.12.1 task Construct . 161 2.12.2 taskloop Construct . 166 2.12.3 taskloop simd Construct . 171 2.12.4 taskyield Construct . 173 2.12.5 Initial Task . 174 Contents iii 2.12.6 Task Scheduling . 175 2.13 Memory Management Directives . 177 2.13.1 Memory Spaces . 177 2.13.2 Memory Allocators . 178 2.13.3 allocate Directive . 181 2.13.4 allocate Clause . 184 2.14 Device Directives . 186 2.14.1 Device Initialization . 186 2.14.2 target data Construct . 187 2.14.3 target enter data Construct . 191 2.14.4 target exit data Construct . 193 2.14.5 target Construct . 197 2.14.6 target update Construct . 205 2.14.7 Declare Target Directive . 210 2.15 Interoperability . 216 2.15.1 interop Construct . 217 2.15.2 Interoperability Requirement Set . 220 2.16 Combined Constructs . 221 2.16.1 Parallel Worksharing-Loop Construct . 221 2.16.2.