Vitis AI User Guide
Total Page:16
File Type:pdf, Size:1020Kb
Vitis AI User Guide UG1414 (v1.3) February 3, 2021 Revision History Revision History The following table shows the revision history for this document. Section Revision Summary 02/03/2021 Version 1.3 Entire document Updated links 12/17/2020 Version 1.3 Entire document Minor changes Deep-Learning Processor Unit Added new topics: Alveo U200/U250: DPUCADF8H, Alveo U50/U50LV/U280: DPUCAHX8L, and Versal AI Core Series: DPUCVDX8G. TensorFlow 2.x Version (vai_q_tensorflow2) Added new section PyTorch Version (vai_q_pytorch) Added new topics: Module Partial Quantization, vai_q_pytorch Fast Finetuning, and vai_q_pytorch Quantize Finetuning. Chapter 5: Compiling the Model Added new section: Compiling with an XIR-based Toolchain. Chapter 10: Integrating the DPU into Custom Platforms Added new chapter. Appendix A: Vitis AI Programming Interface Added new section: VART APIs. 07/21/2020 Version 1.2 Entire document Minor changes 07/07/2020 Version 1.2 Entire document • Added Vitis AI Profiler topic. • Added Vitis AI unified API introduction. DPU Naming Added new topic Chapter 2: Getting Started Updated the chapter 03/23/2020 Version 1.1 DPUCAHX8H Added new topic Entire document Added contents for Alveo U50 support, U50 DPUV3 enablement, including compiler usage and model deployment description. UG1414 (v1.3) February 3, 2021Send Feedback www.xilinx.com Vitis AI User Guide 2 Table of Contents Revision History...............................................................................................................2 Chapter 1: Vitis AI Overview.....................................................................................5 Navigating Content by Design Process.................................................................................... 6 Features........................................................................................................................................7 Vitis AI Tools Overview................................................................................................................7 Vitis AI Containers..................................................................................................................... 22 Minimum System Requirements.............................................................................................23 Development Flow Overview................................................................................................... 23 Chapter 2: Getting Started ..................................................................................... 25 Installation and Setup...............................................................................................................25 Running Examples.................................................................................................................... 36 Support.......................................................................................................................................40 Chapter 3: Understanding the Vitis AI Model Zoo Networks.............. 41 Chapter 4: Quantizing the Model........................................................................ 43 Overview.....................................................................................................................................43 Vitis AI Quantizer Flow..............................................................................................................45 TensorFlow 1.x Version (vai_q_tensorflow)............................................................................ 46 TensorFlow 2.x Version (vai_q_tensorflow2).......................................................................... 58 PyTorch Version (vai_q_pytorch)..............................................................................................65 Caffe Version (vai_q_caffe)....................................................................................................... 76 Chapter 5: Compiling the Model.......................................................................... 82 Vitis AI Compiler........................................................................................................................ 82 Compiling with an XIR-based Toolchain.................................................................................83 Compiling with DPUCADX8G..................................................................................................103 VAI_C Usage............................................................................................................................. 105 Chapter 6: Deploying and Running the Model........................................... 107 Deploying and Running Models on Alveo U200/250.......................................................... 107 UG1414 (v1.3) February 3, 2021Send Feedback www.xilinx.com Vitis AI User Guide 3 Programming with VART........................................................................................................ 107 DPU Debug with VART............................................................................................................ 109 Multi-FPGA Programming...................................................................................................... 115 Apache TVM and Microsoft ONNX Runtime.........................................................................118 Chapter 7: Profiling the Model............................................................................ 120 Vitis AI Profiler......................................................................................................................... 120 Chapter 8: Optimizing the Model...................................................................... 127 Chapter 9: Accelerating Subgraph with ML Frameworks.................... 128 Partitioning Functional API Call in TensorFlow....................................................................128 Partitioner API......................................................................................................................... 128 Partitioning Support in Caffe.................................................................................................130 Chapter 10: Integrating the DPU into Custom Platforms....................132 Appendix A: Vitis AI Programming Interface............................................. 133 VART APIs................................................................................................................................. 133 Appendix B: Legacy DNNDK..................................................................................147 DNNDK N2Cube Runtime.......................................................................................................147 DNNDK Examples ...................................................................................................................149 DNNDK Programming for Edge............................................................................................ 153 DNNDK Utilities....................................................................................................................... 159 Profiling Using the DNNDK Profiler...................................................................................... 166 DNNDK Programming APIs................................................................................................... 170 Appendix C: Additional Resources and Legal Notices........................... 229 Xilinx Resources.......................................................................................................................229 Documentation Navigator and Design Hubs...................................................................... 229 References................................................................................................................................229 Please Read: Important Legal Notices................................................................................. 230 UG1414 (v1.3) February 3, 2021Send Feedback www.xilinx.com Vitis AI User Guide 4 Chapter 1: Vitis AI Overview Chapter 1 Vitis AI Overview The Vitis™ AI development environment accelerates AI inference on Xilinx® hardware platforms, including both Edge devices and Alveo™ accelerator cards. It consists of optimized IP cores, tools, libraries, models, and example designs. It is designed with high efficiency and ease of use in mind to unleash the full potential of AI acceleration on Xilinx FPGAs and on adaptive compute acceleration platforms (ACAPs). It makes it easier for users without FPGA knowledge to develop deep-learning inference applications, by abstracting the intricacies of the underlying FPGA and ACAP. Figure 1: Vitis AI Stack User Application Frameworks Vitis AI Models Model Zoo Custom Models AI Compiler | AI Quantizer | AI Optimizer Vitis AI AI Profiler | AI Library Development Kit Xilinx Runtime library (XRT) Overlay Deep Learning Processing Unit (DPU) X24893-120920 UG1414 (v1.3) February 3, 2021Send Feedback www.xilinx.com Vitis AI User Guide 5 Chapter 1: Vitis AI Overview Navigating Content by Design Process Xilinx® documentation is organized around a set of standard design processes to help you find relevant content for your current development task. This document covers the following design processes: • Machine Learning and Data Science: Importing a machine learning model from a Caffe, Pytorch, TensorFlow, or other popular framework onto Vitis™ AI, and then optimizing and evaluating its effectiveness. Topics in this document that apply to this design process include: • Chapter 2: Getting