Outline • Introduction • Application Partitioning • Generic Networking Equipment Introduction to Network Processors • Network Processor Focus • Network Processor Challenges • Fitting the Architecture to the Problem Space Guest Lecture at UC Berkeley, 07Mar2002 Chuck Narad, Principal System Architect Intel Network Processor Division Introduction to Network Processors 1 Introduction to Network Processors 2 3/7/2002 3/7/2002 Introduction What is a Network Processor? • Overview of networking applications and processing • Terminology emerged in the industry 1997-1998 systems that are tuned to address them – Many startups competing for the network building -block market • Network Processing vs. Network Processors • Broad variety of products are presented as an NP • Discussion of Network Processors must be driven by • Some amount of integration and some amount of what networking applications do programmability – Moving data from here to there: Switching, Routing, • Generally some characteristics that enable efficient Aggregation/Disaggregation, Bridging etc. processing of network headers in cells or packets – Providing services: Security, Monitoring, Traffic Shaping etc. • Value proposition of NP’s: • Sometimes support for higher-level flow management – Improve TTM and to reduce investment by turning a silicon • Wide spectrum of capabilities and target markets design problem into a programming problem – Provide flexibility and field upgradability in networking equipment Introduction to Network Processors 3 Introduction to Network Processors 4 3/7/2002 3/7/2002 Motivations for using a Network Processor What Can an NP Be Used For? • “Flexibility of a fully programmable processor with • Highly dependent on user’s application: performance approaching that of a custom ASIC.” • Faster time to market (no ASIC lead time) • Integrated uP + system controller + “acceleration” – Instead you get software development time • Fast forwarding engine with access to a “slow-path” • Field upgradability leading to longer lifetime for control agent products in the field – Ability to adapt deployed equipment to evolving and • A smart DMA engine emerging standards and new application spaces • An intelligent NIC • Enables multiple products using common hardware • A highly integrated set of components to replace a • Allows the network equipment vendors to focus on bunch of ASICs and the blade control uP their value-add Introduction to Network Processors 5 Introduction to Network Processors 6 3/7/2002 3/7/2002 1 Common Features in NPs NP Architectural Challenges • Pool of multithreaded forwarding engines • Application-specific architecture • Integrated or attached GP uP • Yet, covering a very broad space with varied (and ill- defined) requirements and no useful benchmarks • High Bandwidth and High Capacity Memories • The Swiss Army Knife challenge – Embedded and external SRAM and DRAM – Versatile but does a bad job at everything • Integrated media interface or media bus • Need to understand the environment • Interface to a switching fabric or backplane • Need to understand network protocols • Interface to a “host” control processor • Need to understand networking applications • Interface to coprocessors • Have to provide solutions before the actual problem is defined – Decompose into the things you can know – Flows, bandwidths, “Life-of-Packet” scenarios, specific common functions Introduction to Network Processors 7 Introduction to Network Processors 8 3/7/2002 3/7/2002 Network Application Partitioning • Network processing is partitioned into planes – Forwarding Plane: Data movement, protocol conversion, etc – Control Plane: Flow management, (de)fragmentation, protocol stacks and signaling stacks, statistics gathering, Problem Spaces Addressed by NP’s management interface, routing protocols, spanning tree etc. • Control Plane is sometimes divided into Connection and Management Planes – Connections/second is a driving metric – Often connection management is handled closer to the data plane to improve performance-critical connection setup/teardown – Control processing is often distributed and hierarchical Introduction to Network Processors 9 Introduction to Network Processors 10 3/7/2002 3/7/2002 Network Processor Focus Generic Networking Equipment Line Card LC Control LC Control Line Card • The NP is generally aimed at Forwarding Plane tasks Processor Processor – Data shovel Ingress – Light Touch: Framing, SAR’ing, Classification and Lookups, FP FP Mappings (port, path, tag, flow, etc.) Media Processing Processing Media – High Throughput – Queuing and Scheduling – Backplane encapsulation and decapsulation Fabric or ControlControl • Packets requiring heavier work are offloaded to Backplane ProcessorProcessor Control Plane or Coprocessor • NP’s usually provide a forwarding plane closely FP FP coupled with a uP. Media Media Processing Processing Egress – The microprocessor may implement the entire control plane – May handle a portion of it locally (e.g. flow setup) and have LC Control LC Control an external host which provides the higher-level control Line Card Processor Processor Line Card plane Introduction to Network Processors 11 Introduction to Network Processors 12 3/7/2002 3/7/2002 2 L3-L7 Application Examples Oversimplified Categorization of Applications • Some or all packets require involvement of a GPP – Handles exceptions, manages connections, or handles higher layer processing • Examples of L3 processing: Payload Inspection Real Time – IP Fragment reassembly, IP filtering, MPOA, LANE, Multicast Virus Scanning Forwarding, Virtual Private Networks (IPSEC) TCP Header Virtual Private Network • Examples of L4 processing: Firewall IP Header – Proxying, Port Mapping (NAT), TCP stream following, stream Load Balancing reassembly, content-based routing,QoS/CoS, Rate Shaping, Ethernet Network Monitoring Load balancing (LB) Header Application Processing Complexity • Examples of L5-L7 processing: Packet Inspection Complexity Quality of Service Routing – Content-based load balancing (CBLB), RMON-2, traffic engineering, accounting, Intrusion Detection, Virus Detection Switching • Many/most higher-layer functions implicitly include forwarding (routing). Introduction to Network Processors 13 Introduction to Network Processors 14 3/7/2002 3/7/2002 Categorizing Application Types and Needs More Detailed Application Characteristics • Applications can: – be high- or low-touch on packet data Application Data State Compute CP – be high- or low-touch on application state touch touch touch – span a spectrum of compute needs from low-compute to very Switching Low Low/Med Low/Med Low compute-intensive Routing Low Low/Med Low/Med Low • Some applications are high touch for a percentage of packets: QoS Low/Med Low/Med Low/Med Low – where one or more packets require high {packet, state} touch and Stateful Firewall Low/Med Low/Med Low-High Med/High Proxy Firewall Med/high Med Med High relatively high compute to establish flow state, and Load Balancing Med Med/High Low/Med Med/High – subsequent packets in that flow require simple forwarding CB Load Balance High Med/High Low/Med High • The simplest of L4 applications can be high-touch or -compute VPN High Med High – TCP/UDP checksums require touch of entire packet Virus Detection High High High High • Some modern MAC’s do this per datagram frag for you, minor math to combine IDS High High High High – IP fragment reassembly, TCP flow assembly require creation and management of flow state, and copies or linked lists of buffers in order to do processing on streams of packets rather than per-pkt Introduction to Network Processors 15 Introduction to Network Processors 16 3/7/2002 3/7/2002 Basic Paradigm of Forwarding Plane Processing Canonical Network Processing Flow • Examine header(s) • Do lookup(s) Buffer & Descriptor – e.g. bridging tables, IPv4 LPM, flow identification table Recovery • Select and Execute Actions Classification – Packet (or cell) modifications Results Packet RX Modifications, … – Application State modifications (tables, counters, flow records) Inspect Connections, Steer – Queuing Queuing, – Possibly heavy lifting such as connection management, crypto, Lookups etc RegEx string search… • Transmit may also include scheduling/shaping Tables State Schedule Modifications • Since ingress and egress are typically on different Application blades, “TX” and “RX” may be to/from the fabric State • Housekeeping: Buffers and descriptors must be Reports to Admin portion TX allocated and recovered for each frame Introduction to Network Processors 17 Introduction to Network Processors 18 3/7/2002 3/7/2002 3 Challenges for NPs • Infinitely variable problem space • “Wire speed”; small time budgets per cell/packet • Poor memory utilization; fragments, singles – Mismatched to burst-oriented memory Challenges Faced by Network Processors • Poor locality, sparse access patterns, indirections – Memory latency dominates processing time – New data, new descriptor per cell/packet. Caches don’t help – Hash lookups and P-trie searches cascade indirections • Random alignments due to encapsulation – 14-byte Ethernet headers, 5-byte ATM headers, etc. – Want to process multiple bytes/cycle • Short-lived flows (esp. HTTP) => high rate of “exceptions” • Sequentiality requirements within flows; sequencing overhead/locks • Shared data structures -> locks; latency and contention costs Introduction to Network Processors 19 Introduction to Network Processors 20 3/7/2002 3/7/2002 Processing Time Budgets Latency Challenges of Packet Processing • Packet