Paolo Narvaez Sr. Principal Engineer, Engineering Director for Analytics and AI Solutions Enterprise and Government Organization XLDB April 3, 2019 CLOUD COMMUNICATION ENTERPRISE SERVICE SERVICE AND PROVIDERS PROVIDERS GOVERNMENT
DRIVE THE BEST EXPERIENCE & SLA PREPARING FOR 5G SILOED APPLICATIONS & DATA PACKETS
PROTECT DATA PRIVACY NETWORK TRANSFORMATION SLOW DEPLOYMENTS OF NEW SERVICES
DRIVE HIGHER REVENUE DEMANDS AT THE NETWORK EDGE SECURITY EXPLOITS GROWING
REDUCE DOWNTIME NEW SERVICE OPPORTUNITIES / RPU DATA MOVEMENT & NETWORK BOTTLENECKS
DRIVE OPERATIONAL EFFICIENCY CONVERGED WORKFLOWS & INFRASTRUCTURE
INTEL DATA CENTER GROUP MOVE | STORE | PROCESS Move Faster STORE MORE PROCESS EVERYTHING
ETHERNET INTEL® XEON® SCALABLE PROCESSORS
SILICON PHOTONICS INTEL® XEON® D PROCESSORS
INTEL ATOM® C PROCESSORS OMNI-PATH FABRIC DC SERIES SSD QLC 3D NAND DRIVE INTEL® FPGAS
INTEL® NERVANA™ NNP
INTEL® MOVIDIUS™ TECHNOLOGY
HARDWARE SOFTWARE AND FASTER TIME TO ENHANCED SYSTEM-LEVEL SECURITY OPTIMIZED VALUE
INTEL DATA CENTER GROUP MOVE | STORE | PROCESS Analytics Artificial Hybrid Network HPC Intelligence Cloud Transformation Microsoft SQL Server* Universal Customer BigDL on Apache Spark* Microsoft Azure Stack* Simulation & Modeling Business Operations Premises Equipment
Red Hat OpenShift* Professional Microsoft SQL Server AI Inferencing NFVi: Red Hat* Enterprise Data Container Platform Visualization Warehouse Windows Server* VMware vSAN* NFVi: Ubuntu* Genomics Analytics
Microsoft Windows Microsoft SQL Server NFVi: FusionSphere* HPC AI Converged Enterprise Data Server Software Defined* Warehouse Visual Cloud Linux* Huawei FusionStorage* Delivery Network
SAP HANA* Blockchain: Hyperledger Fabric* 1H’19 Solution Refresh - Existing Workloads
1H’19 Solutions on NEW Workloads
INTEL DATA CENTER GROUP MOVE | STORE | PROCESS INTEL.COM/SELECTSOLUTIONS EMBEDDED UNINTERRUPTED HARDWARE ENHANCED BUILT-IN LEADERSHIP WORKLOAD GROUNDBREAKING ARTIFICIAL INTELLIGENCE ENHANCED VALUE PERFORMANCE MEMORY INNOVATION ACCELERATION SECURITY AGILITY & UTILIZATION
INTEL DATA CENTER GROUP MOVE | STORE | PROCESS INTEL.COM/XEONSCALABLE Advanced performance SCALABLE PERFORMANCE Featuring intel® speed select technology (3 in 1)
SUPPORT FOR 2.0TB & 4.5 TB SUPPORT FOR 48 3.8 2.3 71.5 350 28 4.0 2.2 38.5 165 DDR4 MEMORY 24 3.9 2.4 35.75 165 9242 CORES TURBO BASE CACHE TDP 8276 CORES TURBO BASE CACHE TDP CAPACITY SUPPORT 8260Y CORES TURBO BASE CACHE TDP SKUS AVAILABLE
SUPPORT FOR 2.0TB & 4.5TB SUPPORT FOR 32 3.7 2.3 71.5 250 24 3.9 2.4 35.7 165 DDR4 MEMORY 18 3.9 2.6 24.75 150 9222 CORES TURBO BASE CACHE TDP CAPACITY SUPPORT 8260 CORES TURBO BASE CACHE TDP 6240Y CORES TURBO BASE CACHE TDP SKUS AVAILABLE
32 3.7 2.1 71.5 250 SUPPORT FOR 9221 CORES TURBO BASE CACHE TDP 16 3.0 2.2 35.7 165 12 3.2 2.2 16.5 85 8253 CORES TURBO BASE CACHE TDP 4214Y CORES TURBO BASE CACHE TDP Optimized for highest per-core SCALABLE performance SUPPORT FOR NETWORKING/NFV specialized SUPPORT FOR 2.0TB & 4.5TB 24 3.7 2.1 35.75 150 28 4.0 2.7 38.5 205 DDR4 MEMORY 6252 CORES TURBO BASE CACHE TDP 8280 CORES TURBO BASE CACHE TDP CAPACITY SUPPORT SUPPORT FOR SKUS AVAILABLE 24 3.6 2.3 35.75 150 SUPPORT FOR 6252N CORES TURBO BASE CACHE TDP SUPPORT FOR 20 3.9 2.5 27.5 150 26 4.0 2.7 35.75 205 6248 CORES TURBO BASE CACHE TDP 8270 CORES TURBO BASE CACHE TDP SUPPORT FOR 20 3.5 2.3 27.5 125 SUPPORT FOR 2.0TB & 4.5TB 6230N CORES TURBO BASE CACHE TDP DDR4 MEMORY SUPPORT FOR 18 3.9 2.6 24.75 150 24 3.9 2.9 35.75 205 6240 CORES TURBO BASE CACHE TDP CAPACITY SUPPORT 8268 CORES TURBO BASE CACHE TDP SKUS AVAILABLE 16 3.9 2.3 22 105 SUPPORT FOR SUPPORT FOR 2.0TB & 4.5TB 5218N CORES TURBO BASE CACHE TDP SUPPORT FOR 22 3.7 2.1 30.25 140 DDR4 MEMORY 16.5 105 CAPACITY SUPPORT 24 3.9 3.8 6238 CORES TURBO BASE CACHE TDP 8256 CORES TURBO BASE CACHE TDP SKUS AVAILABLE VM density VALUE specialized SUPPORT FOR SUPPORT FOR 18 4.0 3.1 24.75 200 20 3.9 2.1 27.5 125 SUPPORT FOR 6254 CORES TURBO BASE CACHE TDP 6230 CORES TURBO BASE CACHE TDP 24 3.6 1.9 33 135 6262V CORES TURBO BASE CACHE TDP
SUPPORT FOR SUPPORT FOR 12 4.2 3.3 24.75 165 18 3.9 2.2 24.75 125 SUPPORT FOR 6246 CORES TURBO BASE CACHE TDP 5220 CORES TURBO BASE CACHE TDP 20 3.6 1.8 27.5 115 6222V CORES TURBO BASE CACHE TDP SUPPORT FOR 8 4.4 3.6 24.75 150 16 3.9 2.3 22 125 SUPPORT FOR 6244 CORES TURBO BASE CACHE TDP 5218 CORES TURBO BASE CACHE TDP Long-life cycle and nebs-thermal FRIENDLY
SUPPORT FOR 16 3.9 2.8 22 150 16 3.2 2.1 16.5 100 SUPPORT FOR 6242 CORES TURBO BASE CACHE TDP 22 3.7 1.9 30.25 125 4216 CORES TURBO BASE CACHE TDP 6238T CORES TURBO BASE CACHE TDP Available processor options SUPPORT FOR 8 4.0 3.3 24.75 130 12 3.2 2.2 16.5 85 SUPPORT FOR 6234 CORES TURBO BASE CACHE TDP 4214 CORES TURBO BASE CACHE TDP 20 3.9 2.1 27.5 125 UP TO 6230T CORES TURBO BASE CACHE TDP Large DDR memory tier SUPPORT 4.5TB UP TO SUPPORT FOR Medium DDR memory tier SUPPORT 2TB 12 3.7 2.7 19.25 125 SUPPORT FOR 6226 CORES TURBO BASE CACHE TDP 10 3.2 2.2 13.75 85 NETWORKING & NFV specialized 4210 CORES TURBO BASE CACHE TDP 18 3.9 1.9 24.75 105 5220T CORES TURBO BASE CACHE TDP
Search value specialized SUPPORT FOR 4 3.9 3.8 16.5 105 SUPPORT FOR thermal & long-life cycle support 5222 CORES TURBO BASE CACHE TDP 8 3.2 2.1 11 85 4208 CORES TURBO BASE CACHE TDP 16 3.8 2.1 22 105 VM density value specialized 5218T CORES TURBO BASE CACHE TDP Intel® speed select TECHNOLOGY 8 3.7 3.0 16.5 115 SUPPORT FOR 5217 CORES TURBO BASE CACHE TDP 6 1.9 1.9 8.25 85 3204 CORES TURBO BASE CACHE TDP 8 3.2 2.2 11 70 4209T CORES TURBO BASE CACHE TDP Turbo MAXIMUM INTEL® turbo boost technology 2.0 SUPPORT FOR 2.0TB & 4.5TB frequency (in GHz) 10 3.4 2.5 16.5 85 DDR4 MEMORY 5215 CORES TURBO BASE CACHE TDP CAPACITY SUPPORT Base Base frequency (in GHz) SKUS AVAILABLE Search application VALUE specialized
Cache Processor cache (in MB) SUPPORT FOR SUPPORT FOR Tdp Thermal design power (in WATTS) 8 3.5 2.5 16.5 85 18 3.9 2.7 24.75 125 4215 CORES TURBO BASE CACHE TDP 5220S CORES TURBO BASE CACHE TDP RCP recommended customer pricing ($ US DOLLARS) nfV Network function virtualization ALL INFORMATION PROVIDED IS SUBJECT TO CHANGE WITHOUT NOTICE. INTEL MAY MAKE CHANGES TO SPECIFICATIONS AND Vm Virtual machine PRODUCT DESCRIPTIONS AT ANY TIME, WITHOUT NOTICE. PLEASE VISIT INTEL.COM/XEON OR CONTACT YOUR INTEL REPRESENTATIVE nebs Network equipment-building system TO OBTAIN THE LATEST INTEL PRODUCT SPECIFICATIONS. © COPYRIGHT 2019. INTEL CORPORATION. Leadership XEON High Analytics & HIGH PERFORMANCE Performance Artificial DENSITY computing intelligence INFRASTRUCTURE
G UP UP UP H UP T TO112 TO2X TO3.8 Z TO3 B CORES MORE COMPUTE INTEL® TURBO BOOST DDR4-2933 Mt/S 2S SYSTEM DENSITY TECHNOLOGY 2.0 2S SYSTEM
INTEL DATA CENTER GROUP MOVE | STORE | PROCESS INTEL.COM/XEONSCALABLE WORLD RECORDS AND COUNTING…
G UP UP H UP T UP TO DDR4 8+ TO4.4 Z TO36B TO2X /s INTEL® TURBO BOOST SYSTEM MEMORY SYSTEM MEMORY 2933 MT SOCKETS TECHNOLOGY 2.0 In a eight socket system CAPACITY & 16 Gb DIMMs USING INTEL® OPTANE™ DC COMPARED TO INTEL® XEON® IN a SYSTEM PERSISTENT MEMORY AND DRAM PLATINUM 8180 PROCESSOR
95 World Records featuring Intel® Xeon® processors as of September 14, 2018. https://newsroom.intel.com/news/intel-xeon-scalable-processors-set-95-new-performance-world-records/ INTEL DATA CENTER GROUP MOVE | STORE | PROCESS INTEL.COM/XEONSCALABLE Business resilience with HARDWARE-ENHANCED SECURITY Agile service delivery with enhanced EFFICENCY INTEL® DEEP LEARNING BOOST MITIGATIONS FOR SIDE-CHANNEL methods • New integrated AI acceleration for inference • Enhanced performance over software-only mitigations INTEL® Speed select TECHNOLOGY • Configurable core/frequency processor attributes Encryption + Accelerators • Prioritize workload performance • Supports platform TCO optimizations • Available with Intel® QuickAssist Technology • Integrated Intel® Advanced Vector Extensions 512 • Intel® Optane™ DC persistent memory on-module INTEL® INFRASTRUCTURE MANAGEMENT TECHNOLOGIES data encryption • Industry-leading Intel® Virtualization Technology • Seamless VM migration for over 5 generations INTEL® Security libraries • Enhanced Intel® Resource Director Technology • New Intel resource orchestration software • New Intel® Threat Detection Technology • New Intel® Ethernet 800 series with Application Device • Intel® Trusted Execution Technology Queues (ADQ) and Dynamic Device Personalization (DDP) • Intel® Cloud Integrity Technology • New Intel® Optane™ DC D4800X (Dual Port) SSDs
INTEL DATA CENTER GROUP MOVE | STORE | PROCESS INTEL.COM/XEONSCALABLE Intel® Ethernet Intel® Ethernet Intel® Ethernet 500 Series 700 Series 800 Series Niantic Fortville Columbiaville1 More Queue and Steering Hardware Assists . Application Device Queues (ADQ)
Fully Programmable Pipeline Partially Programmable Pipeline . Table definition with DDP profile packages . Table definition modifications with a Dynamic Device Personalization (DDP) Storage profile package . RDMA (iWARP* & RoCE*v2) Fixed Pipeline
Intel® Ethernet Adaptive Virtual Functions Capability Richness Capability SR-IOV and VMDq (Intel® AVF) Less 10GbE 40GbE 100GbE
1FeaturesINTEL & scheduleDATA CENTERare subject toGROUP change. All products, computer systems, dates and figures specified are preliminary based on current expectations, and are subject to change without notice. MOVE | STORE | PROCESS INTEL.COM/ETHERNET Intel innovation for ultimate agility and flexibility
High-Performance compute Any-to-any integration Any Developer
CACHE-COHERENT INTEL® XEON® MEMORY, ANALOG, LOGIC INTEL® QUARTUS® PRIME PROCESSOR ACCELERATION DESIGN SOFTWARE FOR ANY NODE, SUPPLIER, IP HARDWARE DEVELOPERS MASSIVE BANDWIDTH RAPID EASIC-BASED OPTIMIZATION ONEAPI FOR SOFTWARE DEVELOPERS
INTEL DATA CENTER GROUP MOVE | STORE | PROCESS INTEL.COM/FPGA INTRODUCING
SIZE AND DATA PERSISTENCE FAST MEMORY OF STORAGE Enhance data insights by Redefining
Supported on future the Memory & Storage Hierarchy Intel® Xeon® Scalable Processors Platinum and Gold SKUs
INTEL DATA CENTER GROUP Intel Confidential – NDA Use Only MOVE | STORE | PROCESS 12 Memory DRAM HOT TIER Persistent Memory
Improving Storage Performance Gap Storage SSD performance 3D NAND SSD WARM TIER Delivering Intel® QLC 3D Nand SSD efficient storage Cost performance GAp
HDD / TAPE COLD TIER
INTEL DATA CENTER GROUP MOVE | STORE | PROCESS INTEL.COM/OPTANE PERSISTENT PERFORMANCE AFFORDABLE MEMORY CAPACITY & MAXIMUM CAPACITY FOR MANY APPLICATIONS
APPLICATION APPLICATION
VOLATILE MEMORY POOL OPTANE PERSISTENT DRAM DRAM AS CACHE MEMORY
OPTANE PERSISTENT MEMORY
INTEL DATA CENTER GROUP MOVE | STORE | PROCESS Intel Persistent Memory THE FAST PERFORMANCE OF MEMORY WITH THE AVAILABITY AND CAPACITY OF STORAGE NEW USER/APPLICATION APP / WORKLOAD
KERNEL/OS FILE SYSTEM & DRIVER
MEMORY/STORAGE DRAM INTEL® OPTANE™ DC INTEL® 3D NVMe TYPE SSDS & SATA SAS & SATA
SMALL CAPACITY LARGE CAPACITY CHARACTERISTICS FAST PERFORMANCE SLOW PERFORMANCE
I/O CONTROLLER
PROCESSING Future Intel® Xeon® Scalable Processor
It’s MEMORY. It’s STORAGE. It’s BOTH. FLEXIBLE AND SCALABLE TO ACCELERATE YOUR WORKLOAD’S DEMANDS AND DATA INSIGHTS
DATA CENTER GROUP Intel Confidential – NDA Use Only 15 MEMORY DRAM HOT TIER PERSISTENT INTEL® OPTANE™ SSDDC D4800X DUAL-PORT MEMORY • PERFORMANCE + RESILIENCY FOR CRITICAL ENTERPRISE IT APPS • DUAL PORT CONNECTIONS ENABLE 24X7 DATA AVAILABILITY WITH REDUNDANT, HOT SWAPPABLE DATA PATHS
STORAGE Intel® 3D Nand SSDs INTEL® SSDD-5 P4326 E1.L • COST-OPTIMIZED, ENABLES GREATER WARM STORAGE • NEW E1.L FORM FACTOR SCALABLE TO ~1PB (IN 1U)
HDD / TAPE COLD TIER
INTEL DATA CENTER GROUP MOVE | STORE | PROCESS INTEL.COM/OPTANE BEST FIT IS SHOWN, BOTH PRODUCTS MAY BE VIABLE
storage Infrastructure database AI / analytics hpc COMMS
RDMA/Replication Memory Caching/Persistence Real Time Analytics Content Delivery Network (CDN) Oracle Exadata* VMWare ESXi* SAP HANA* SAS* MSFT Hyper-V* MS-SQL* Comms SP custom KVM* Oracle Exadata* SDS Redis Labs* Aerospike* Machine Learning Analytics Memcached* Redis* Ceph* Apache Spark* VDI RocksDB*
Hyper-Converged (HCI) Scratch & IO Nodes Storage memory HPC Flex Memory VMware vSAN VMWare ESXi* Microsoft S2D MSFT Hyper-V* Nutanix* KVM* Cisco HyperFlex*
INTEL DATA CENTER GROUP MOVE | STORE | PROCESS INTEL.COM/OPTANE THREE SOLUTION EXAMPLES 2ND GEN INTEL® XEON® SCALABLE + INTEL® OPTANE™ DC PERSISTENT MEMORY
SQL Server Database, TimesTen
18 DELIVER MORE FOR LESS WITH INTEL® OPTANE™ DC PERSISTENT MEMORY + SAP HANA
3 TB DRAM + 6 TB Intel® More capacity Optane™ DC persistent memory= 9TB TOTAL
Go faster Restart time Restart time Faster
MINIMIZE DOWNTIME 20 mins 90 seconds 13xrestart1 SAP HANA SAP SAVE MORE Cost Cost / DB Terabyte ~$62,495USD ~$38,357 USD 39%savings2
CPU: 4 Intel® Xeon® Platinum 8280 Processor CPU: 4 Intel® Xeon® Platinum 8280M Processor MEMORY: 48 x 128GB DDR4 for 6 TB system MEMORY: 24 x 128GB DDR4 + 24 x 256GB Intel Optane DC PMEM for 9TB system
Performance results are based on testing as of Jan 30, 2019 and may not reflect all publicly available security updates. No product or component can be absolutely secure. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to www.intel.com/benchmarks. 1. For detailed configs and pricing see slide 13. Columnar store entire reload into DRAM for 1.3TB data set is 20 mins. Entire system restart before is 32 Minutes and with DCPMM it is 13.5 Minutes (12 mins for OS + 1.5 mins) Pricing Guidance as of March 1, 2019. Intel does not guarantee any costs or cost reduction. You should consult other information and performance tests to assist you in your purchase decision.
19 withother For more products.complete information visit vary. You should consult information other you to assistandperformance evaluating testsin fully your contemplated purchase as component can Performance on testingresultsarebased Pricing Guidance of as March 1, 2019. Intel does not guarantee any costs or cost reduction. You should consult other informa SYSmark WITH 2 FASTER QUERY TODAY EXPERIENCE CUSTOMER EXPERIENCE CUSTOMER PAST and be absolutely absolutely be secure. MobileMark ND , computerspecific aremeasured usingcomponents, systems,software,and operations functions. Any change to any of GEN INTEL at 1TB scale factor with a 2S Intel Xeon E5 Xeon Intel a 2S with factor scale 1TB at Software With 2S Intel Xeon Platinum 8280 processors: 8280 Platinum Xeon Intel With 2S PERFORMANCE WITH LATEST SW & HW & SWLATEST WITH PERFORMANCE by Intel FASTER DATA WAREHOUSEQUERY DATAFASTER and workloads andworkloads in performancemay testsusedhave performance for onl beenoptimized 903,302 as 33,681 of of 2/8/2019 PERFORMANCE www.intel.com/benchmarks v3 (4 at 1TB scale factor scale 1TB at 26.8 ® XEON andmay available configuration No See product all updates. securitypublicly for details.not reflect disclosures yr queries per hour per queries old system) old queries per hour queries X better ® PROCESSOR + SQL SERVER*+ SQL PROCESSOR 1 . 1 - 2 Configuration:See - 2699 WITH MORE USERS slide 13, 14 13, slide tio n and performance tests to assist you in your purchase decision. 22 30% INCREASE VMS PER NODE FOR MULTI FOR INCREASEVMS PER NODE memory at memory TENANT VIRTUALIZED DATABASES VIRTUALIZED TENANT 30 Microsoft SQL VM instances withinstances DRAM VM SQL Microsoft s, 36% ~$1,108 including theperformance of that product Microsoft SQL VM instances at instances VM SQL Microsoft lower estimated HW cost perVM HW cost estimated lower y on Intelmicroprocessors.Performance such tests, ~$1,588 more VMs node VMs more per thosefactors may causeto theresults USD perVM USD USD per VM USD per 2 when combined or or - 3
21 SQL SERVER product whenproduct combinedFor withother more products.complete information visit to vary. resultsYou should consult information other you to assistandperformance evaluating testsin fully your contemplat as Software product Performance on testingresultsarebased TIMESTEN AND WITH ORACLE*DATABASE MINIMIZE DOWNTIME & IMPROVE INSIGHTS TODAY EXPERIENCE CUSTOMER EXPERIENCE CUSTOMER PAST SYSmark or or component can and workloads andusedworkloads in performancemay testshavefor performance beenoptimized only on Intelmicroprocessors. Performa and MobileMark Efficiency measured measured by transactions Efficiency With 2nd Gen Intel Xeon Scalable Xeon Intel 2nd Gen With Wholesaler Wholesaler supply managingchain be absolutely absolutely be secure. , computerspecific aremeasured usingcomponents, systems,software,and operations functions. Any change to any of thosefa warehouse Wholesaler can now manage manage Wholesaler nowcan OPERATIONAL EFFICIENCYOPERATIONAL WITH ORACLE WITH ORACLE DATABASE with with 2S 5 year old system 3.7x 3.7x more TPMs per per minute (TPM) 3.77 processors: IMPROVED IMPROVED by - to Intel X - store store inventory better as of October October 2018 1 and may not reflect all publicly andmaypublicly available all updates securitynot reflect Copy database Copy database persistent fromimage < 1 second restarttime > 10 mintime restart www.intel.com/benchmarks storage storage into volatile DRAM WITH ORACLE WITH ORACLE TIMESTEN 1.35 TB database with Database Database persistent is FASTER IN 2.7 with database TB DB START UPDB START ~1 second - MEMORY MEMORY 2 . 1 - 3 Configuration:See ed ed Immediate persistence, no write to storage . See configuration disclosures for details. No configuration No See for details..disclosures purchases,including theperformance of that buffer written synchronously to storage Average throughput Average throughput Transactions commit to buffer…then Log buffer in persistent buffer persistent memoryLog in THROUGHPUT & DURABILITY& THROUGHPUT slides Log buffer buffer volatile Log in DRAM nce suchtests, WITH ORACLE WITH ORACLE BETTER TRANSACTION 16,17 6.49 X better = = TIMESTEN 1.16M TPS 176K cto 3 rsmay causethe TPS
24 ORACLE DB & TIMESTEN INTEL OPTIMIZATION F O R CAFFE RESNET - 50 NEW AI ACCELERATION
INTEL® XEON® PLATINUM 9200 PROCESSORS W I T H I N T E L ® DL BOOST VECTOR NEURAL NETWORK INSTRUCTION 11 For BUILT-IN inference Acceleration
INTEL® XEON® PLATINUM 8200 PROCESSORS W I T H I N T E L ® DL BOOST 10 Optimizations for Developers & Data Scientists
Toolkit INTEL® XEON® PLATINUM 8100 PROCESSORS 5.7X 9 optimized Frameworks INFERENCE1.0 THROUGHPUT8 (IMAGES/SEC) Jul’17 Dec’18 Apr’19 Data type
Performance results are based on testing as of dates shown in configuration and may not reflect all publicly available security updates. Configurations and benchmark details can be found on slide/page 52. No product or component can be absolutely secure. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks. INTEL DATA CENTER GROUP MOVE | STORE | PROCESS AI.INTEL.COM Making inference go faster: vector instructions
• AVX 512: Advanced Vector x-86: first launched in 2013 (!) – Xeon Phi
• Vector arithmetic => perfect for deep neural networks
• AVX 512 VNNI: multiply & accumulate 64 8- bit values -> 32 bit result with a single instruction
• 4x more operations than 32bit
• ¼ the memory vs. 32bit
• In practice: roughly 2x speedup
28 Accelerated Inference: 2nd Gen Xeon Scalable Processor
Intel has a new vector neural network instruction (VNNI) to extend the Intel® AVX-512 instruction. Available on all 2nd Generation Intel® Xeon® Scalable Processors, VNNI speeds up dense computations characteristic of convolutional neural networks (CNNs) and deep neural networks (DNNs).
2nd generation Intel®AVX-512: with Intel® Deep Learning Boost Accelerating ai/dl inference for: Intel® Deep Learning Boost Input INT8 Output INT32 Input
1st generation Intel®AVX-512: without Intel® Deep Learning Boost Language translation Object detection
Input Output Output Output INGT8 INT16 INT32 INT32
Input Constant Constant INT8 INT16 INT32 And more!
29 29 Optimized Deep Learning Frameworks and Toolkits Gen on Gen Performance gains for ResNet-50 with Intel® DL Boost
2S Intel® Xeon® Platinum 8280 Processor vs 2S Intel® Xeon® Platinum 8180 Processor
Intel® Xeon® 2nd Gen Intel® Scalable Xeon® Scalable Processor Processor
INT8 w/ FP32 3.0x 3.7x 3.9x 4.0x 3.9x Intel® DL Boost
INT8 w/ INT8 Intel® DL Boost 1.8x 2.1x 1.8x 2.3x 1.9x
See Configuration Details 5 Performance results are based on testing as of dates shown in configuration and may not reflect all publicly available security updates. No product can be absolutely secure. See configuration disclosure for details. Optimization Notice: Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit: http://www.intel.com/performance
30 Intel® Select Solution for AI Inferencing Jump start your AI strategy with low latency and high throughput inferencing
High Inference Low Latency Accelerate Time from Intel® Optane™ Performance SSDs and Intel® to Inference with 2nd generation Ethernet Network with use of the Intel® Intel® Xeon® Scalable Adapters Distribution of processors featuring Intel® OpenVINO™ toolkit Deep Learning Boost
Learn more at intel.com/selectsolutions 31 Intel® Select Solution for
AI Inferencing Configuration
Hardware Base – single node Plus – single node Option 1 Plus – single node Option 2 Software Version
CPU 2 x Intel® Xeon® Scalable CPU 2 x Intel® Xeon® Scalable CPU 2 x Intel® Xeon® Scalable CPU OpenVINO Toolkit 2018 R5 Runtime 6248 (CLX Gold) 8268 (CLX Platinum) 6248 (CLX Gold) OpenVINO Model 0.4 Memory 192 GB 384 GB 384 GB Server (min) (12 x 16 GB 2666MHz DDR4 ECC (12 x 32 GB 2666MHz DDR4 ECC (12 x 32 GB 2666MHz DDR4 ECC TensorFlow 1.12 RDIMM) RDIMM) RDIMM) PyTorch 1.0.1 Boot Drive 1 x 256GB Intel® SSD DC P4101 1 x 256GB Intel® SSD DC P4101 1 x 256GB Intel® SSD DC P4101 (M.2 80mm PCIe 3.0 x4, 3D2, TLC (M.2 80mm PCIe 3.0 x4, 3D2, TLC (M.2 80mm PCIe 3.0 x4, 3D2, TLC MXNet 1.3.1 ) or higher ) or higher ) or higher Intel Python 2019 Update 1 0.17 (implied by Storage Data Drive: 3.2 TB (NVMe Intel Data Drive: IntelPlus DC P4610 Config Series Data Options Drive: Intel DC P4610 Series MKL-DNN SSD P4610) 1.6TB 15mm U.2 NVMeComingSSD 1.6TB Q2 15mm U.2 NVMe SSD OpenVINO) Cache Drive: 375 GB (NVMe Cache Drive: Intel DC P4800X Cache Drive: Intel DC P4800X Minimum Performance Benchmark P4800) [Optane] Series 375gb U.2 NVMe SSD Series 375gb U.2 NVMe SSD Standards
Accelerator Intel® Programmable Acceleration OpenVINO Toolkit Card with Intel Arria® 10 GX FPGA At least 2000 images per second with Top-5 accuracy of 25 GbE, dual port 25 GbE, dual port 25 GbE, dual port Data ResNet 50 91% Network With ImageNet TensorFlow Framework Plus Option 1: Plus Option 2: At least 1300 images per Xeon Platinum Higher Memory second with Top-5 accuracy of Higher Memory FPGA 91%
Required elements bolded. 32 Complete configuration details to be documented in Reference Design – ETA March 2019. Intel® Select Solution for AI Inferencing Jump start your AI strategy with low latency and high throughput inferencing
Minimum Throughput Minimum Throughput
AT LEAST AT LEAST *All verified solution 2000* 1300* providers will New 2nd IMAGES PER SECOND1 IMAGES PER SECOND2 meet or exceed Generation DURING INFERENCE USING RESNET-50 DURING INFERENCE USING RESNET-50 these values. AND TOP-5 ACCURACY OF 91% AND TOP-5 ACCURACY OF 91% INCLUDES INCLUDES 3.75x 3.77x IMPROVEMENT USING IMPROVEMENT USING INTEL® DEEP LEARNING BOOST INTEL® DEEP LEARNING BOOST WITH MODEL CONVERSION TO INT-8 WITH MODEL CONVERSION TO INT-8
Learn more at intel.com/selectsolutions 33 Intel® Math Kernel Library for Deep Neural Networks Intel® Machine Learning Scalable Library Optimized Libraries Intel® Data Analytics Acceleration Library Intel® Integrated Performance Primitives
TensorFlow*, PyTorch* Optimized AI Frameworks Caffe*, PaddlePaddle* MXNET*, …
Threading Building Blocks Intel® MPI Library Parallel Programming FRAMEWORK OpenMP*
Intel® Compilers (LLVM, GCC, Fortran) Compilers nGraph
Intel® VTune™ Amplifier Profiling tools Intel® Advisor Intel® Inspector
Intel® Parallel Studio XE Intel® System Studio Intel® Distribution of OpenVINO toolkit Toolkits / Intel® Distribution for Python* Data Plane Development Kit Persistent Memory Development Kit Tool Suites Intel® Resource Director Technology (OWCA & Intel® PRM) Storage Performance Development Kit
OWCA: ORCHESTRATION-AWARE WORKLOAD COLLOCATION AGENT INTEL® PRM: INTEL® PLATFORM RESOURCE MANAGER INTEL DATA CENTER GROUP MOVE | STORE | PROCESS SOFTWARE.INTEL.COM NOTICES & DISCLAIMERS
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No product or component can be absolutely secure. Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. For more complete information about performance and benchmark results, visit http://www.intel.com/benchmarks . Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit http://www.intel.com/benchmarks . Intel® Advanced Vector Extensions (Intel® AVX)* provides higher throughput to certain processor operations. Due to varying processor power characteristics, utilizing AVX instructions may cause a) some parts to operate at less than the rated frequency and b) some parts with Intel® Turbo Boost Technology 2.0 to not achieve any or maximum turbo frequencies. Performance varies depending on hardware, software, and system configuration and you can learn more at http://www.intel.com/go/turbo. Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction.
Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether referenced data are accurate.
Intel, the Intel logo, and Intel Xeon are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as property of others.
© 2019 Intel Corporation.
INTEL DATA CENTER GROUP MOVE | STORE | PROCESS NOTICES & DISCLAIMERS
FTC Optimization Notice If you make any claim about the performance benefits that can be achieved by using Intel software developer tools (compilers or libraries), you must use the entire text on the same viewing plane (slide) as the claim.
Optimization Notice: Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice Revision #20110804
INTEL DATA CENTER GROUP MOVE | STORE | PROCESS