Programmer’s Guide Release 18.02.2 June 15, 2018 CONTENTS 1 Introduction 1 1.1 Documentation Roadmap...............................1 1.2 Related Publications..................................2 2 Overview 3 2.1 Development Environment..............................3 2.2 Environment Abstraction Layer............................4 2.3 Core Components...................................4 2.4 Ethernet* Poll Mode Driver Architecture.......................6 2.5 Packet Forwarding Algorithm Support........................6 2.6 librte_net........................................6 3 Environment Abstraction Layer7 3.1 EAL in a Linux-userland Execution Environment..................7 3.2 Memory Segments and Memory Zones (memzone)................ 11 3.3 Multiple pthread.................................... 12 3.4 Malloc.......................................... 14 4 Service Cores 19 4.1 Service Core Initialization............................... 19 4.2 Enabling Services on Cores............................. 19 4.3 Service Core Statistics................................ 20 5 Ring Library 21 5.1 References for Ring Implementation in FreeBSD*................. 22 5.2 Lockless Ring Buffer in Linux*............................ 22 5.3 Additional Features.................................. 22 5.4 Use Cases....................................... 22 5.5 Anatomy of a Ring Buffer............................... 22 5.6 References....................................... 31 6 Mempool Library 33 6.1 Cookies......................................... 33 6.2 Stats.......................................... 33 6.3 Memory Alignment Constraints............................ 33 6.4 Local Cache...................................... 34 6.5 Mempool Handlers.................................. 35 6.6 Use Cases....................................... 36 7 Mbuf Library 37 i 7.1 Design of Packet Buffers............................... 37 7.2 Buffers Stored in Memory Pools........................... 39 7.3 Constructors...................................... 39 7.4 Allocating and Freeing mbufs............................. 39 7.5 Manipulating mbufs.................................. 39 7.6 Meta Information.................................... 39 7.7 Direct and Indirect Buffers.............................. 41 7.8 Debug......................................... 42 7.9 Use Cases....................................... 42 8 Poll Mode Driver 43 8.1 Requirements and Assumptions........................... 43 8.2 Design Principles................................... 44 8.3 Logical Cores, Memory and NIC Queues Relationships.............. 45 8.4 Device Identification, Ownership and Configuration................ 45 8.5 Poll Mode Driver API................................. 49 9 Generic flow API (rte_flow) 54 9.1 Overview........................................ 54 9.2 Flow rule........................................ 54 9.3 Rules management.................................. 74 9.4 Isolated mode..................................... 78 9.5 Verbose error reporting................................ 79 9.6 Helpers......................................... 79 9.7 Caveats......................................... 80 9.8 PMD interface..................................... 80 9.9 Device compatibility.................................. 81 9.10 Future evolutions................................... 83 9.11 API migration...................................... 83 10 Traffic Metering and Policing API 88 10.1 Overview........................................ 88 10.2 Configuration steps.................................. 88 10.3 Run-time processing................................. 88 11 Traffic Management API 90 11.1 Overview........................................ 90 11.2 Capability API..................................... 90 11.3 Scheduling Algorithms................................ 91 11.4 Traffic Shaping..................................... 91 11.5 Congestion Management............................... 91 11.6 Packet Marking.................................... 92 11.7 Steps to Setup the Hierarchy............................. 92 12 Wireless Baseband Device Library 94 12.1 Design Principles................................... 94 12.2 Device Management................................. 94 12.3 Device Operation Capabilities............................ 96 12.4 Operation Processing................................. 98 12.5 Sample code...................................... 100 13 Cryptography Device Library 103 ii 13.1 Design Principles................................... 103 13.2 Device Management................................. 103 13.3 Device Features and Capabilities.......................... 105 13.4 Operation Processing................................. 107 13.5 Symmetric Cryptography Support.......................... 109 13.6 Sample code...................................... 112 13.7 Asymmetric Cryptography.............................. 115 14 Security Library 116 14.1 Design Principles................................... 116 14.2 Device Features and Capabilities.......................... 119 15 Rawdevice Library 125 15.1 Introduction....................................... 125 15.2 Design......................................... 125 16 Link Bonding Poll Mode Driver Library 127 16.1 Link Bonding Modes Overview............................ 127 16.2 Implementation Details................................ 128 16.3 Using Link Bonding Devices............................. 135 17 Timer Library 138 17.1 Implementation Details................................ 138 17.2 Use Cases....................................... 139 17.3 References....................................... 139 18 Hash Library 140 18.1 Hash API Overview.................................. 140 18.2 Multi-process support................................. 141 18.3 Implementation Details................................ 141 18.4 Entry distribution in hash table............................ 142 18.5 Use Case: Flow Classification............................ 143 18.6 References....................................... 144 19 Elastic Flow Distributor Library 145 19.1 Introduction....................................... 145 19.2 Flow Based Distribution................................ 145 19.3 Example of EFD Library Usage........................... 149 19.4 Library API Overview................................. 150 19.5 Library Internals.................................... 151 19.6 References....................................... 154 20 Membership Library 155 20.1 Introduction....................................... 155 20.2 Vector of Bloom Filters................................ 156 20.3 Hash-Table based Set-Summaries.......................... 159 20.4 Library API Overview................................. 161 20.5 References....................................... 163 21 LPM Library 164 21.1 LPM API Overview.................................. 164 21.2 Implementation Details................................ 164 iii 22 LPM6 Library 168 22.1 LPM6 API Overview.................................. 168 22.2 Use Case: IPv6 Forwarding............................. 172 23 Flow Classification Library 173 23.1 Overview........................................ 173 24 Packet Distributor Library 180 24.1 Distributor Core Operation.............................. 181 24.2 Worker Operation................................... 182 25 Reorder Library 183 25.1 Operation........................................ 183 25.2 Implementation Details................................ 183 25.3 Use Case: Packet Distributor............................. 184 26 IP Fragmentation and Reassembly Library 185 26.1 Packet fragmentation................................. 185 26.2 Packet reassembly................................... 185 27 Generic Receive Offload Library 188 27.1 Overview........................................ 188 27.2 Two Sets of API.................................... 188 27.3 Reassembly Algorithm................................ 189 27.4 TCP/IPv4 GRO.................................... 190 27.5 VxLAN GRO...................................... 190 28 Generic Segmentation Offload Library 192 28.1 Overview........................................ 192 28.2 Limitations....................................... 192 28.3 Packet Segmentation................................. 193 28.4 Supported GSO Packet Types............................ 194 28.5 How to Segment a Packet.............................. 195 29 The librte_pdump Library 197 29.1 Operation........................................ 197 29.2 Implementation Details................................ 198 29.3 Use Case: Packet Capturing............................. 198 30 Multi-process Support 199 30.1 Memory Sharing.................................... 199 30.2 Deployment Models.................................. 200 30.3 Multi-process Limitations............................... 202 31 Kernel NIC Interface 203 31.1 The DPDK KNI Kernel Module............................ 203 31.2 KNI Creation and Deletion.............................. 205 31.3 DPDK mbuf Flow................................... 205 31.4 Use Case: Ingress................................... 205 31.5 Use Case: Egress................................... 205 31.6 Ethtool......................................... 206 31.7 Link state and MTU change.............................. 206 iv 32 Thread Safety of DPDK Functions 207 32.1 Fast-Path APIs..................................... 207 32.2 Performance Insensitive API............................. 208 32.3 Library Initialization.................................. 208 32.4 Interrupt Thread...................................