SYCL Command Queues Per SYCL Platform Device ▪ Can Create Multiple Command Queues for a Host Device, E.G

Agenda

• What is SYCL and why Intel needs it? • SYCL language/API • SYCL programming model overview • Kernel execution model • “Hello world” example • Language evolution • DPC++/SYCL Compilers • App compilation and execution flow • SYCL implementations • References

- A modern platform includes: - One or more CPUs - One or more GPUs - DSP processors - FPGA - Individual processors have many (possibly heterogeneous) cores Programmers want to write a single portable program that uses ALL resources in the heterogeneousIntel platform IceLake architecture Optimization Notice Copyright © 2019, Intel Corporation. All rights reserved. 3 *Other names and brands may be claimed as the property of others. What is SYCL?

Single-source heterogeneous programming using STANDARD C++ ▪ Use C++ templates and lambda functions for host & device code Aligns the hardware acceleration of OpenCL with direction of the C++ standard

Developer Choice The development of the two specifications are aligned so code can be easily shared between the two approaches

C++ Kernel Language Single-source C++ Low Level Control Programmer Familiarity ‘GPGPU’-style separation of Approach also taken by device-side kernel source C++ AMP and OpenMP code and host code

Reactive to OpenCL Pros and Cons: Proactive about Future C++: • OpenCL has a well-defined, portable • SYCL is based on purely modern C++ execution model. and should feel familiar to C++11 users. • OpenCL is prohibitively verbose for • SYCL expected to run ahead of C++Next many application developers. regarding heterogeneity and parallelism. ISO C++ of tomorrow may • OpenCL remains a C API and only look a lot like SYCL. recently supported C++ kernels. • Not held back by C99 or C++03 • Just-in-time compilation model and compatibility goals. disjoint source code is awkward and contrary to HPC usage models.

Terminology: SYCL platform

A SYCL implementation consists of: SYCL Platform ▪ A Host: Makes SYCL API calls

▪ SYCL Devices: Runs SYCL Kernels Host ▪ The Host and SYCL Devices make up an SYCL Platform ▪ One system may have multiple installed SYCL Platforms Example: Intel Platform (CPU) + NVIDIA Platform (GPU) CPU SYCL Device GPU SYCL Device ▪ SYCL Host Device is native C++ implementation of SYCL API on the host Some Other FPGA SYCL Device SYCL Device

Applications create SYCL Command Queues per SYCL Platform Device ▪ Can create multiple Command Queues for a Host Device, e.g. one per thread ▪ No cross-device Command Queues, e.g. for GPU Queue automatic load balancing CPUCPU Queue Queue

Applications execute SYCL Command Groups via CPU SYCL Device GPU SYCL Device Command Queues ▪ Example Commands: Copy Memory, Fill Buffers, Launch Kernels, etc. Some Other FPGA SYCL Device SYCL Device

• Explicit ND-range for control • 1, 2 or 3 dimensional range • Kernel code is SPMD/SIMT ND-range

Work-group

Work-item Global work space

GEN11 slice architecture diagram

• Additional level of execution model hierarchy - access SIMD unit • GEN EU can execute 16 FP32 floating point operations per clock • [2 ALUs x SIMD-4 x 2 Ops (Add + Mul)]

Sub-group decomposition of work-group

Work-group

Sub-group

Work-item Global work space

1. Basic data parallel parallel_for( num_work_items )

2. Explicit ND-Range parallel_for( NDRange( global_size, work_group_size ) )

▪ Enable SPMD/SIMT coding for OpenCL and CUDA experts

parallel_for_work_group( num_work_groups ) { 3. Hierarchical parallelism parallel_for_work_item( items_per_work_group ) } ▪ Exploit scope to control execution granularity

▪ Similar to Kokkos hierarchical parallelism 4. Single task single_task() {}

parallel_for_work_group (…) { … parallel_for_work_item (…) { … } }

▪ Fundamentally top down expression of parallelism ▪ Many embedded features and details, not covered here

• Private Memory Private Private Private Private Memory Memory Memory Memory –Per Work Item (kernel invocation) Work-Item Work-Item Work-Item Work-Item • Local Memory Local Memory Local Memory

–Shared within a Work Group Workgroup Workgroup

• Global/Constant Memory Global/Constant Memory –Visible to all Work Groups Compute Device • Host Memory Host Memory –On the Host Host Memory management is explicit. We must explicitly move data from host -> global -> local and back.

Supported features Unsupported features ₊ templates ₋ dynamic memory allocation ₊ classes ₋ dynamic polymorphism ₊ operator overloading ₋ runtime type information ₊ static polymorphism ₋ exception handling ₊ lambdas ₋ function pointers* ₊ short vector types (2/3/4/8/16-wide) ₊ reach library of built-in functions ₋ pointer structure members* ₋ static variables*

void vector_add(const std::vector &A, const std::vector &B, const std::vector &C) { ... } Let’s compute C = A + B on GPU!

void vector_add(const std::vector &A, const std::vector &B, const std::vector &C) { Step 1: create buffers sycl::buffer d_A{ A.data(), A.size() }; sycl::buffer d_B{ B.data(), B.size() }; (represent both host and sycl::buffer d_C{ C.data(), C.size() }; device memory)

... }

void vector_add(const std::vector &A, const std::vector &B, const std::vector &C) { sycl::buffer d_A{ A.data(), A.size() }; sycl::buffer d_B{ B.data(), B.size() }; sycl::buffer d_C{ C.data(), C.size() }; Step 2: create a device queue sycl::gpu_selector device_selector; (developer can specify a device type via sycl::queue q(device_selector); device selector or use default selector) ... }