Oneapi Single Programming Model to Deliver Cross-Architecture Performance Industry Initiative, Intel® Oneapi Beta Products

oneAPI Single Programming Model to Deliver Cross-Architecture Performance Industry initiative, Intel® oneAPI Beta Products

Edmund Preiss

One Intel Software & Architecture (OISA) Organization

Refer to software.intel.com/articles/optimization- notice for more information regarding performance & optimization choices in Intel software products. Copyright ©, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. Programming Challenges for Multiple Architectures Application Workloads Need Diverse Hardware Scalar Vector Matrix Spatial Growth in specialized workloads Middleware / Frameworks Variety of data-centric hardware required

No common programming language or APIs Language & Libraries Inconsistent tool support across platforms XPUs Each platform requires unique software investment CPU GPU FPGA Other accel.

Refer to software.intel.com/articles/optimization- notice for more information regarding performance & optimization choices in Intel software products. Copyright ©, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. 1 introducing oneapi Application Workloads Need Diverse Hardware Scalar Vector Matrix Spatial Unified programming model to simplify development across diverse architectures Middleware / Frameworks Unified and simplified language and libraries for expressing parallelism Industry Intel Uncompromised native high-level language performance Initiative Product

Based on industry standards and open specifications XPUs Interoperable with existing HPC programming models CPU GPU FPGA Other accel.

oneAPI Industry Specification A standards based cross-architecture language, DPC++, based on C++ and SYCL Direct Programming API-Based Programming

Powerful APIs designed for acceleration of key domain-specific functions Data Parallel C++ Libraries Low-level hardware interface to provide a hardware abstraction layer to vendors Low-Level Hardware Interface

Open standard to promote community and XPUs industry support Enables code reuse across architectures and vendors CPU GPU FPGA Other accel.

Visit oneapi.com for more details

Some capabilities may differ per architecture and custom-tuning will still be required. Refer to software.intel.com/articles/optimization- notice for more information regarding performance & optimization choices in Intel software products. Copyright ©, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. 3 Data parallel C++ Standards-based, Cross-architecture Language

Parallelism, productivity and performance for CPUs and Accelerators Direct Programming: Allows code reuse across hardware targets, while permitting custom tuning for a Data Parallel C++ specific accelerator Open, cross-industry alternative to single architecture proprietary language Community Extensions Based on ISO C++ and Khronos SYCL Delivers C++ productivity benefits, using common and familiar C and C++ constructs Khronos SYCL Incorporates SYCL from the Khronos Group to support data parallelism and heterogeneous programming

Community Project to drive language enhancements ISO C++ Extensions to simplify data parallel programming Open and cooperative development for continued evolution

The open source and Intel beta DPC++ compiler currently supports hardware including Intel CPUs, GPUs, and FPGAs. Codeplay announced a DPC++ compiler that targets Nvidia GPUs.

DPC++ Designed for acceleration of key domain-focused functions Math Threading Library

Analytics/ DNN ML Comm Each can be custom-coded for any platform to deliver ML uncompromised performance Video Processing

Custom-tuning for each architecture will still be required. Refer to software.intel.com/articles/optimization- notice for more information regarding performance & optimization choices in Intel software products. Copyright ©, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. 5 OneAPI initiative –Ecosystem support

UNIVERSITY OF Indian Institute of CAMBRIDGE Technology Delhi

These organizations support the oneAPI initiative ‘concept’ for a single, unified programming model for cross -architecture development. It does not indicate any agreement to purchase or use of Intel’s products. *Other names and brands may be claimed as the property of others. Intel‘s oneAPI Procucts

intel oneAPI Toolkits and Libraries

Optimization Notice Copyright © 2019, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. What’s Inside Intel® Parallel Studio XE Software Development Tool Suite - for HPC and Enterprise and ISVs Cluster Edition Composer Edition Professional Edition BUILD ANALYZE SCALE Compilers & Libraries Analysis Tools Cluster Tools Intel® Math Kernel Library Intel® VTune™ Amplifier Intel® MPI Library C / C++, Performance Profiler Message Passing Interface Library Intel® Data Analytics Fortran Acceleration Library Compilers Intel® Inspector Intel® Trace Analyzer & Collector Intel Threading Building Blocks Memory & Thread Debugger MPI Tuning & Analysis C++ Threading Intel® Advisor Intel® Cluster Checker Intel® Integrated Performance Primitives Vectorization Optimization Cluster Diagnostic Expert System Image, Signal & Data Processing Thread Prototyping & Flow Graph Analysis Intel® Distribution for Python* High Performance Python

Operating System: Windows*, Linux*, MacOS1* Intel® Architecture Platforms

1Available only in the Composer Edition.

8 Next Generation of Commercial Intel® SW Development Tool Products

Intel oneAPI supports Intel XPUs (Intel CPUs, Intel discrete GPUs , Intel FPGAs , ...) Intel® ONEAPI products(beta) Application Workloads Optimized Middleware & Frameworks

Intel® oneAPI Product Distributed through a core toolkit and a complementary set of add-on Direct API-Based domain-specific toolkits Programming Programming Compatibility Analysis & tool Data Parallel Debug Tools Includes DPC++ compatibility tool for Libraries C++ code migration along with advanced performance analysis and debug tools Low-Level Hardware Interface

Beta Available Now XPUs

CPU GPU FPGA Other accel.

Visit software.intel.com/oneapi for more details

Some capabilities may differ per architecture and custom-tuning will still be required. Other accelerators to be supported in the future. Optimization. Notice Copyright © 2019, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. 10 Intel® ONEAPI toolkits(beta)

Toolkits Tailored to Your Needs: Native Code | Data Scientists & AI | Systems

Native Code Developers, start with the Intel® oneAPI Base Toolkit. Intel® oneAPI Base Toolkit A core set of high-performance tools for building Data Parallel C++ applications and oneAPI library based applications Learn More

Add-on Domain-specific Toolkits for Specialized Workloads

Intel® oneAPI HPC Toolkit Intel® oneAPI IoT Toolkit Intel® oneAPI DL Framework Intel® oneAPI Rendering Developer Toolkit Toolkit Deliver fast C++, Fortran, & OpenMP* Building high-performing, efficient, Build deep learning frameworks or Create high-performance, high- applications that scale reliable solutions that run at the customize existing ones so fidelity visualization applications network’s edge applications run faster

Learn More Learn More Learn More Learn More

Toolkits Powered by oneAPI: Data Scientists & AI Toolkits Systems To olkit

Intel® AI Analytics Toolkit Intel® Distribution of OpenVINO™ Intel® System Bring-Up Toolkit Accelerate E2E machine learning & Toolkit data science pipelines with optimized Deploy high performance inference & Debug & tune systems for power & DL frameworks & high-performing applications from edge to cloud performance Python libraries. (production-level tool)

Learn More Learn More Learn More

Intel® oneAPI Base Toolkit Core set of frequently used tools and libraries for developing high-performance applications across Direct Programming API-Based Programming Analysis & debug Tools diverse architectures—CPU, GPU, FPGA. Intel® oneAPI DPC++ Intel® oneAPI Intel® Compiler DPC++ Library VTune™ Profiler Who Uses It? Intel® DPC++ Intel® oneAPI Math Kernel Library Intel® Advisor A broad range of developers across industries Compatibility Tool Add-on toolkit users since this is the base for all toolkits Intel® Distribution Intel® oneAPI for Python* Data Analytics Library GDB*

Intel® FPGA Add-on Intel® oneAPI Top Features/Benefits for oneAPI Base Threading Building Toolkit Blocks Data Parallel C++ compiler, library, and analysis tools Intel® oneAPI Video DPC++ Compatibility tool helps migrate existing code written Processing Library in CUDA* Intel® oneAPI Python distribution includes accelerated scikit-learn, NumPy, Collective Comms. SciPy libraries Library Intel® oneAPI Optimized performance libraries for threading, math, data Deep Neural Network analytics, deep learning, and video/image/signal processing Library Intel® Integrated Performance Primitives

Compiler to deliver uncompromised parallel programming productivity and performance across CPUs and accelerators Allows code reuse across hardware targets, while permitting custom tuning for a specific accelerator Open, cross-industry alternative to single architecture proprietary language

DPC++ is based on C++ and SYCL Delivers C++ productivity benefits, using common and familiar C and C++ constructs Incorporates SYCL from The Khronos Group to support data parallelism and heterogeneous programming Builds upon Intel’s decades of experience in architecture and high performance compilers

There will still be a need to tune for each architecture. Refer to software.intel.com/articles/optimization- notice for more information regarding performance & optimization choices in Intel software products. Copyright ©, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. 14 Intel® DPC++ Compatibility tool(beta) minimizes code migration time

Assists developers migrating code written in CUDA* to DPC++ once, generating human readable code wherever possible

~80-90% of code migrates automatically

Inline comments are provided to help developers complete their code

Optimized C++ Standard Algorithms Contains 75 parallelized C++17 algorithms and utilities for efficient application development & deployment on a variety of hardware

Based on parallel libraries that C++ developers are already familiar with Incorporates popular libraries Parallel STL and Boost.Compute for easier developer adoption

Integrated with Intel® DPC++ Compatibility Tool Complements all oneAPI DPC++ components to simplify migration of developers’ CUDA* code to DPC++ code

Boost media and video application performance with hardware-accelerated codecs & programmable graphics on Intel® CPUs & Intel GPUs

Simple API that works the same on CPU & GPU

Using the API, developers have full control over codec visual quality & performance

Helps developers create high performance deep learning frameworks

Abstracts out instruction set & other complexities of performance optimizations

Same API for both Intel CPUs and GPUs, use the best technology for the job

Supports Linux*, Windows*

Open sourced for community contributions

oneCCLoneCCL API API

oneCCLoneCCL Provides optimized communication patterns OFIOFI API API oneCCLoneCCL Intel®Intel Software SW for high performance on Intel® CPUs & GPUs to MPIMPI Collective Collective APIAPI rd distribute model training across multiple nodes 3rd 3party party software SW SPIRSPIR--VV & & Level Level00 OFIOFI API HardwHW are Intel®Intel MPI MPI Library

Transparently supports many interconnects, API/ControlAPI/Control such as Intel® Omni-Path Architecture, DataData Libfabric DPC++ InfiniBand*, & Ethernet DPC++ Runtime (Open Fabrics Interface) Runtime

Built on top of lower-level communication EFA middleware⎯ MPI & libfabrics Driver/Kernel verbs sockets psm2 driver driver

Enables efficient implementations of collectives

used for deep learning training⎯ all-gather, HWHardware EFA RoCE iWarp Infiniband Ethernet OPA GPU all-reduce, & reduce-scatter

NetworkNetwork

Refer to software.intel.com/articles/optimization- notice for more information regarding performance & optimization choiceschoices in in Intel software products. Copyright ©, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. 19 Intel® VTune™ Profiler DPC++ Profiling-Tune for CPU, GPU & FPGA

Analyze Data Parallel C++ (DPC++) See the lines of DPC++ that consume the most time

Tune for CPU, GPU & FPGA Optimize for any supported hardware accelerator

Optimize Offload Tune OpenMP* offload performance

Wide Range of Performance Profiles CPU, GPU, FPGA, threading, memory, cache, storage… Supports Popular Languages DPC++, C, C++, Fortran, Python*, Go*, Java*, or a mix

There will still be a need to tune for each architecture. Refer to software.intel.com/articles/optimization- notice for more information regarding performance & optimization choices in Intel software products. Copyright ©, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. 20 Intel® Advisor Design assistant — Design for Modern Hardware

Offload Advisor Estimate performance of offloading to an accelerator

Roofline Analysis Optimize CPU/GPU code for memory and compute

Vectorization Advisor Add and optimize vectorization

Threading Advisor Add effective threading to unthreaded applications Flow Graph Analyzer Create and analyze efficient flow graphs

There will still be a need to tune for each architecture. Refer to software.intel.com/articles/optimization- notice for more information regarding performance & optimization choices in Intel software products. Copyright ©, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. 21 Intel-enhanced GDB* debugger DPC++ Debug −Heterogeneous Application Debug

High-level language debug support

Multiple accelerator support: CPU, GPU, FPGA emulation

Auto-detect accelerator architecture during application runtime

Non-proprietary open-source solution based on GDB*

A toolkit that makes it easier to build, analyze, optimize & scale HPC applications for Intel® Intel oneAPI Tools for HPC Xeon® Scalable, Intel® Core™ processors & Direct Programming API-Based Programming Analysis Tools Intel® Accelerators. Intel® C++ Compiler Intel® MPI Library with OpenMP* Intel® Inspector

Who Uses It? Intel® Fortran Compiler Intel® oneAPI Intel® Trace C/C++, Fortran, OpenMP & MPI application with OpenMP* DPC++ Library Analyzer & Collector Intel® oneAPI developers Intel® oneAPI Intel® Cluster Checker DPC++ Compiler Math Kernel Library

Intel® DPC++ Intel® oneAPI Intel® VTune™ Profiler Compatibility Tool Data Analytics Library Top Features/Benefits Intel® Distribution Intel® oneAPI Threading for Python* Building Blocks Intel® Advisor Optimized compilers & performance libraries for Intel® FPGA Add-on for Intel® oneAPI Video Intel® architectures oneAPI Base Toolkit Processing Library GDB*

Intel® oneAPI Collective Powerful analysis tools to identify optimization Communications25 opportunities for threading, memory & offloading Library New Stuff! Intel® oneAPI Deep Neural Network Library Standards-driven to scale forward & preserve Parallel Studio Components development investment Intel® oneAPI HPCToolkit + Intel® Integrated Performance Primitives Intel® oneAPI BaseToolkit Transition from Intel Parallel Studio XE to oneAPI Intel® C++ & Fortran Compilers(beta) High Performance Compilers with OpenMP*

Deliver Industry-leading C/C++ & Fortran code performance with OpenMP*, unleash the power of the latest Intel® platforms

Develop optimized & vectorized code for Intel® architectures, including Intel® Xeon® processors

Leverage latest language & OpenMP standards, & compatibility with leading compilers & IDEs

Intel oneAPI Rendering & Ray Tracing Libraries A set of 5 powerful, open source rendering libraries that deliver high-performance, high-fidelity, extensible, & efficient visualization Intel® Embree High-Performance, Feature-rich applications & solutions on Intel® platforms. Ray Tracing & Photorealistic Rendering Digital Domain & Chaos V-Ray Who Uses It? Developers working on high-performance, high-fidelity Intel® OSPRay Scalable, Portable, Distributed visualization applications Image courtesy of Bureau of Economic Geology at Rendering API Univ. of Texas /Austin & Moroccan National Office of Key Usages Hydrocarbon & Mining Creation of studio animation/visual effects content & HPC scientific visualization Intel® Open Image Denoise AI-Accelerated Denoiser for Superior Visual Quality Top Features/Benefits Deliver high-performance, high quality images with global Intel® Open Volume Kernel illumination across all compute infrastructure (on-premise & public) Library Render & Simulate 3D Spatial Access to all system memory space for largest data sets Data Processing Smoke volume, data courtesy OpenVDB example repository Cost efficient, interactive performance for any size data, scale solutions across laptops, workstations, enterprise, HPC & cloud Intel® OpenSWR High-Performance, Scalable, OpenGL*-compatible Rasterizer

Refer to software.intel.com/articles/optimization- notice for more information regarding performance & optimization choices in Intel software products. Copyright ©, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others. 28 Intel® AI Analytics Toolkit(beta) powered by oneapi A toolkit that helps accelerate end-to-end machine learning & data science pipelines with optimized DL frameworks & high-performing Python libraries

Who Uses It? AI researchers & application developers, data scientists

Key Usages AI Research & applications across Finance, Retail, E-commerce, Robotics, Transportation & more

Top Features/Benefits Accelerate end-to-end AI and Data Science pipelines with optimized Python tools built using Intel® oneAPI Libraries

Provides high performance for deep learning training and inference with Intel-optimized TensorFlow and PyTorch Drop-in acceleration for data science workflows from preprocessing through machine learning Scale-out efficiently using the high-performing Python packages, such as NumPy, Scikit-learn, XGBoost and more Supports cross-architecture development and compute (Intel CPUs & future Xe/GPU architecture)

A development sandbox to develop, test Evaluate Workloads and run your workloads across a range of Intel CPUs, GPUs, and FPGAs using Build Heterogenous Applications Intel’s oneAPI beta software

software.intel.com/devcloud/oneapi Prototype your project

No downloads | No hardware acquisition | No installation | No set-up & configuration Get up & running in seconds!

Diverse workloads are driving the need for heterogeneous compute architectures

oneAPI unifies & simplifies programming of heterogeneous architectures delivering developer productivity & performance

oneAPI is an open industry initiative & an Intel reference product No downloads | No hardware acquisition oneAPI is interoperable with existing node & cluster programming model No installation | No set-up & configuration Get Started – software.intel.com/devcloud/oneapi Test code & workloads using the Intel® DevCloud

software.intel.com/devcloud/oneapi

This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps. The products and services described may contain defects or errors known as errata which may cause deviations from published specifications. Current characterized errata are available on request. No product or component can be absolutely secure. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks. INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS”. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTU AL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY , RELATING TO THIS INFORMATION INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGE MENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.

Optimization Notice Intel’s compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice revision #20110804