COM-5503SOFT IP Protocol Stack for 10Gbe, VHDL Source Code Overview
Total Page:16
File Type:pdf, Size:1020Kb
COM-5503SOFT IP/TCP CLIENTS/UDP/ARP/PING STACK for 10GbE VHDL SOURCE CODE OVERVIEW Overview simple enough to interface with any Ethernet MAC component with minimum glue logic. 10Gigabit-speed IP protocols like TCP/IP and UDP/IP can demand a high level of computation on Wireshark Libpcap network capture files can be processors. The trend has been to move the used as receiver input for simulation purposes. implementation of these fast but highly repetitive tasks to a TCP offload engine (TOE) to free the application processor from frequent interrupts. Block Diagram DHCP The COM-5503SOFT is a generic Internet protocol CLIENT stack (including the VHDL source code) designed to support near 10Gbps throughputs on any low- IGMP cost FPGAs running at 156.25 MHz. APPLICATION NEIGHBOR INTERFACE The modular architecture of VHDL components DISCOVERY reflects the various internet protocols implemented STREAMS TO 1 ARP PACKETS within: TCP clients , UDP transmit, UDP receive, REPLY ARP, NDP, PING, IGMP (for multicast UDP) and MAC INTERFACE DHCP client. Ancillary components are also PING UDP_TX included for streaming. These components can be 10Gb MAC REPLY Receive easily enabled or disabled as needed by the user's Interface application. ROUTING TABLE PACKET PARSING The VHDL source code is fully portable to a variety ARP/NDP of FPGA platforms. REQUEST The maximum number of concurrent TCP PACKETS TO UDP_RX connections can be adjusted prior to VHDL STREAMS synthesis depending on the available FPGA resources. TCP_RXBUF TCP CLIENTS The code is written specifically for IEEE 802.3 Ethernet packet encapsulation (RFC 894). It TCP_TXBUF supports IPv4, IPv6, jumbo frames. TCP_TX The code interfaces seamlessly with the COM-5501SOFT 10Gbps Ethernet MAC for the 10Gb MAC Arbitration MAC / PHY layers implementation or the Transmit COM-5401SOFT 10/100/1000 Mbps Ethernet Interface MAC. However, the MAC interface is generic and 190010 1 See COM-5502SOFT for TCP servers. MSS • 845 Quince Orchard Boulevard Ste N • Gaithersburg, Maryland 20878-1676 • U.S.A. Telephone: (240) 631-1111 Facsimile: (240) 631-1676 www.ComBlock.com © MSS 2021 Issued 1/23/2021 Target Hardware The code is written in generic VHDL so that it can be ported to a variety of FPGAs capable of running at 156.25 MHz or above. It was developed and tested on Xilinx 7-series FPGAs. Device Utilization Summary (Excludes 10G Ethernet MAC and XAUI) Device: Xilinx Artix-7 UDP-only: 1 UDP rx, 1 UDP tx, 0 TCP client, ARP, Ping, routing table, IPv4 only, 8KB UDP tx buffer Flip Flops 3125 LUTs 2737 36Kb block RAM 7.5 DSP48 0 TCP IPv4 only: 0 UDP rx, 0 UDP tx, 1 TCP client, ARP, Ping, routing table, IPv4 only, 32KB TCP buffers Flip Flops 2224 LUTs 2381 36Kb block RAM 9.5 DSP48 0 TCP IPv4 only: 0 UDP rx, 0 UDP tx, 2 TCP clients, ARP, Ping, routing table, IPv4 only, 32KB TCP buffers Flip Flops 2735 LUTs 2563 36Kb block RAM 17 DSP48 1 UDP rx, 1 UDP tx, 1 TCP client, ARP, Ping, NDP, routing table, IPv4. IPv6, 32KB TCP buffers Flip Flops 7208 LUTs 9120 36Kb block RAM 32.5 DSP48 0 2 TCP Throughput The TCP throughput is primarily a function of the tx/rx buffers sizes and of the two-way delay. For example, if the two way delay (NIC + FPGA) is 90us Buffers sizes TCP throughput 2kB 133 Mbits/s 8kB 673 Mbits/s 32kB 2.8 Gbits/s 64kB 5.56 Gbits/s 128kB 9.3 Gbits/s 256kB 9.3 Gbits/s If the two-way delay is only 45us, the same TCP throughput can be achieved with half-sized buffers. The buffer size is determined prior to synthesis by the generic parameters TCP_TX/RX_WINDOW_SIZE Throughput Performance Examples UDP IPv4 UDP throughput using 512-Byte data frames: 8.64 Gbits/s IPv4 UDP throughput using 2048-Byte data frames: 9.62 Gbits/s TCP IPv4 TCP single server, uni-directional stream, MTU = 1500 Bytes, equal length maximum size IP frames: 9.38 Gbits/s TCP IPv6 TCP single server, uni-directional stream, MTU = 1500 Bytes, equal length maximum size IP frames: 9.23 Gbits/s IPv4 TCP single server, bi-directional streams, MTU = 1500 Bytes 8.82 Gbits/s in each direction IPv4 TCP single server, uni-directional stream, MTU = 8252 Bytes, equal length maximum size IP frames, buffer size = 32K Bytes: 9.88 Gbits/s IPv6 TCP single server, uni-directional stream, MTU = 8252 Bytes, equal length maximum size IP frames, buffer size = 32K Bytes: 9.86 Gbits/s 3 TCP Latency Performance Examples Interfaces The transmit and receive latency depend on the MAC INTERFACE APP INTERFACE frame length. For a maximum frame length of 1460 CLK UDP_RX_DATA(63:0) SYNC_RESET UDP_RX_DATA_VALID(7:0) bytes, FPGA 156.25 MHz processing clock: MAC_TX_DATA(63:0) UDP RX UDP_RX_SOF MAC_TX_DATA_VALID(7:0) Transmit latency (from the 1st byte of DATA UDP_RX_EOF MAC_TX_SOF MAC TX UDP_RX_FRAME_VALID payload data input to the 1st byte of MAC_TX_EOF DATA payload data output to the Ethernet MAC): MAC_TX_CTS UDP_RX_DEST_PORT_NO_IN(15:0) 23.9µs MAC_TX_RTS CHECK_UDP_RX_DEST_PORT_NO UDP_RX_DEST_PORT_NO_OUT(15:0) Receive latency (from the 1st byte of MAC_RX_DATA(63:0) Ethernet MAC input to the 1st byte of MAC_RX_DATA_VALID(7:0) UDP_TX_DATA(63:0) MAC_RX_SOF MAC RX UDP_TX_DATA_VALID(7:0) payload data output): 12.2µs MAC_RX_EOF DATA UDP_TX_SOF MAC_RX_FRAME_VALID If latency is more important than throughput, the UDP TX UDP_TX_EOF DATA UDP_TX_CTS transmit segmentation threshold can be reduced to UDP_TX_ACK X payload bytes. In this more general case, UDP_TX_NAK UDP_TX_DEST_IP_ADDR(127:0) Transmit latency (from the 1st byte of UDP_TX_DEST_IPv4_6n payload data input to the 1st byte of UDP_TX_DEST_PORTS payload data output to the Ethernet MAC): UDP_TX_SRC_PORT 0.5 + 2X/125 µs TCP_RX_DATA (63:0) The receive latency (from the 1st byte of Ethernet REPLICATED NTCPSTREAMS TCP_RX_DATA_VALID(7:0) MAC input to the 1st byte of payload data TIMES TCP_RX_RTS output): 0.5 + X/125 µs TCP RX TCP_RX_CTS DATA TCP_RX_CTS_ACK TCP_LOCAL_PORTS TCP_TX_DATA(63:0) REPLICATED TCP_TX_DATA_VALID(7:0) NTCPSTREAMS TCP_TX_DATA_FLUSH TIMES TCP TX TCP_TX_CTS CONTROLS DATA TCP_CONNECTED_FLAG MAC_ADDR(47:0) IPv4_ADDR(31:0) TCP_DEST_IP_ADDR IPv4_MULTICAST_ADDR(31:0) TCP_DEST_IPv4_6n IPv4_SUBNET_MASK(31:0) TCP_DEST_PORTS IPv4_GATEWAY_ADDR(31:0) TCP_STATE_REQUESTED IPv6_ADDR(127:0) TCP_STATE_STATUS IPv6_SUBNET_PREFIX_LENGTH(7:0) TCP_KEEPALIVE_EN IPv6_GATEWAY_ADDR(127:0) 180017 Component Interface This interface comprises three primary signal groups: MAC interface (direct connection to COM- 5501SOFT Ethernet MAC or equivalent) TCP streams UDP frames or UDP streams to/from the user application. All signals are clock synchronous. See the clock/timing section. 4 Configuration TCP buffers TCP_TX_WINDOW_SIZE sizes TCP_RX_WINDOW_SIZE The key configuration parameters are brought to the Window size is expressed as interface so that the user can change them 2**n Bytes. Thus a value of 15 dynamically at run-time. Other, more arcane, indicates a window size of parameters are fixed at the time of VHDL synthesis. 32KB. This generic parameter determines how much memory is Pre-synthesis configuration parameters allocated to buffer tcp streams. It applies equally to all concurrent The following configuration parameters are set streams (no individualization). prior to synthesis in the com5503pkg.vhd package Purpose: tradeoff memory or at the top level component com5503.vhd. utilization vs throughput. Configuration Description Memory size ranges from 2KB parameters in (multiple streams/lower com5503pkg.vhd individual throughput) to 1MB Maximum number NTCPSTREAMS_MAX. (single stream/maximum of concurrent TCP This applies to all COM5503 throughput) streams components instantiated in a The window scale option is project. It primarily affects the data recommended on the client side width of the TCP interface. when this server's buffers are Configuration Description larger than 64KB. parameters in DHCP client DHCP_CLIENT_EN com5503.vhd instantiation ‘1’ to instantiate a DHCP client within. DHCP is a protocol used Number of NTCPSTREAMS to dynamically assign IP concurrent TCP This applies to a given COM5503 addresses at power up from streams for a given instance. remote DHCP servers, like a COM5503 Each additional TCP stream gateway. instance. requires additional resources ’0’ when a fixed (static) IP (RAM block, logic). address is defined by the user. IGMP IGMP_EN UDP transmit NUDPTX instantiation instantiated (1) / disabled (0) instantiation instantiated (1) / disabled (0) Enable when using UDP Note: a component handles multicast addresses multiple ports. Inactive input TX_IDLE_TIMEOUT When UDP receive NUDPRX stream timeout segmenting a TCP transmit instantiation instantiated (1) / disabled (0) stream, a packet will be sent out Note: a component handles with pending data if no new data multiple ports was received within the specified Enable IPv6 IPv6_ENABLED timeout. protocols '1' to allow IPv6 protocols in Expressed as integer multiple of addition to the baseline IPv4. 4s. '0' to ignore IPv6 messages. TCP keep- TCP_KEEPALIVE_ MTU size MTU alive period PERIOD Maximum Transmission Unit: IP period in seconds for sending no frame maximum byte size. data keepalive frames. -- Typically 1500 for standard frames, 9000 for jumbo frames. "Typically TCP Keepalives are -- A frame will be deemed invalid sent every 45 or 60 seconds on if its payload size exceeds this an idle TCP connection, and the MTU value. connection is dropped after 3 -- Should match the values in sequential ACKs are missed" MAC layer) -- elastic buffers at the user CLK CLK_FREQUENCY frequency CLK frequency in MHz. Needed interface should be sized to contain at least 4 IP frames payload. See to compute actual delays. ADDR_WIDTH generic parameter. 5 Configuration Description Configuration Description parameters in parameters in ping_10g.vhd stream_2_packets_10g Maximum ping size MAX_PING_SIZE . maximum IP/ICMP size vhd (excluding Ethernet/MAC, Maximum packet size MAX_PACKET_SIZE but including the IP/ICMP when segmenting a When segmenting a header) in 64-bit words. stream to packets transmit stream, a packet Larger echo requests will be will be sent out as soon as ignored.