Washington University in St. Louis Washington University Open Scholarship All Computer Science and Engineering Research Computer Science and Engineering Report Number: WUCSE-2004-72 2004-12-01 Techniques for Processing TCP/IP Flow Content in Network Switches at Gigabit Line Rates David Vincent Schuehler The growth of the Internet has enabled it to become a critical component used by businesses, governments and individuals. While most of the traffic on the Internet is legitimate, a proportion of the traffic includes worms, computer viruses, network intrusions, computer espionage, security breaches and illegal behavior. This rogue traffic causes computer and network outages, reduces network throughput, and costs governments and companies billions of dollars each year. This dissertation investigates the problems associated with TCP stream processing in high-speed networks. It describes an architecture that simplifies the processing of TCP data streams in these environments and presents a... Read complete abstract on page 2. Follow this and additional works at: https://openscholarship.wustl.edu/cse_research Recommended Citation Schuehler, David Vincent, "Techniques for Processing TCP/IP Flow Content in Network Switches at Gigabit Line Rates" Report Number: WUCSE-2004-72 (2004). All Computer Science and Engineering Research. https://openscholarship.wustl.edu/cse_research/1042 Department of Computer Science & Engineering - Washington University in St. Louis Campus Box 1045 - St. Louis, MO - 63130 - ph: (314) 935-6160. This technical report is available at Washington University Open Scholarship: https://openscholarship.wustl.edu/ cse_research/1042 Techniques for Processing TCP/IP Flow Content in Network Switches at Gigabit Line Rates David Vincent Schuehler Complete Abstract: The growth of the Internet has enabled it to become a critical component used by businesses, governments and individuals. While most of the traffic on the Internet is legitimate, a proportion of the traffic includes worms, computer viruses, network intrusions, computer espionage, security breaches and illegal behavior. This rogue traffic causes computer and network outages, reduces network throughput, and costs governments and companies billions of dollars each year. This dissertation investigates the problems associated with TCP stream processing in high-speed networks. It describes an architecture that simplifies the processing of TCP data streams in these environments and presents a hardware circuit capable of TCP stream processing on multi-gigabit networks for millions of simultaneous network connections. Live Internet traffic is analyzed using this new TCP processing circuit. Department of Computer Science & Engineering 2004-72 Techniques for Processing TCP/IP Flow Content in Network Switches at Gigabit Line Rates, Doctoral Dissertation, December 2004 Authors: Schuehler, David V. Abstract: The growth of the Internet has enabled it to become a critical component used by businesses, governments and individuals. While most of the traffic on the Internet is legitimate, a proportion of the traffic includes worms, computer viruses, network intrusions, computer espionage, security breaches and illegal behavior. This rogue traffic causes computer and network outages, reduces network throughput, and costs governments and companies billions of dollars each year. This dissertation investigates the problems associated with TCP stream processing in high-speed networks. It describes an architecture that simplifies the processing of TCP data streams in these environments and presents a hardware circuit capable of TCP stream processing on multi-gigabit networks for millions of simultaneous network connections. Live Internet traffic is analyzed using this new TCP processing circuit. Type of Report: Other Department of Computer Science & Engineering - Washington University in St. Louis Campus Box 1045 - St. Louis, MO - 63130 - ph: (314) 935-6160 WASHINGTON UNIVERSITY SEVER INSTITUTE OF TECHNOLOGY DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING TECHNIQUES FOR PROCESSING TCP/IP FLOW CONTENT IN NETWORK SWITCHES AT GIGABIT LINE RATES by David Vincent Schuehler Prepared under the direction of Professor John W. Lockwood A dissertation presented to the Sever Institute of Washington University in partial fulfillment of the requirements for the degree of Doctor of Science December, 2004 Saint Louis, Missouri WASHINGTON UNIVERSITY SEVER INSTITUTE OF TECHNOLOGY DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING ABSTRACT TECHNIQUES FOR PROCESSING TCP/IP FLOW CONTENT IN NETWORK SWITCHES AT GIGABIT LINE RATES by David Vincent Schuehler ADVISOR: Professor John W. Lockwood December, 2004 Saint Louis, Missouri The growth of the Internet has enabled it to become a critical component used by businesses, governments and individuals. While most of the traffic on the Internet is le- gitimate, a proportion of the traffic includes worms, computer viruses, network intrusions, computer espionage, security breaches and illegal behavior. This rogue traffic causes com- puter and network outages, reduces network throughput, and costs governments and com- panies billions of dollars each year. This dissertation investigates the problems associated with TCP stream processing in high-speed networks. It describes an architecture that simplifies the processing of TCP data streams in these environments and presents a hardware circuit capable of TCP stream processing on multi-gigabit networks for millions of simultaneous network connections. Live Internet traffic is analyzed using this new TCP processing circuit. copyright by David Vincent Schuehler 2004 This dissertation is dedicated to my family, my friends, and the pursuit of knowledge. Contents List of Tables :::::::::::::::::::::::::::::::::::: xi List of Figures ::::::::::::::::::::::::::::::::::: xii Abbreviations :::::::::::::::::::::::::::::::::::: xvii Acknowledgments ::::::::::::::::::::::::::::::::: xx Preface ::::::::::::::::::::::::::::::::::::::: xxii 1 Introduction ::::::::::::::::::::::::::::::::::: 1 1.1 Problem Framework . 4 1.2 Problem Statement . 5 1.3 Contributions . 7 1.4 Organization of Dissertation . 8 2 Background and Motivation :::::::::::::::::::::::::: 11 2.1 Hardware Processing Technologies . 14 2.1.1 Microprocessors . 15 2.1.2 Application Specific Integrated Circuits . 16 2.1.3 Field Programmable Gate Arrays . 16 2.2 Challenges . 18 2.2.1 Performance . 19 2.2.2 Packet Classification . 20 2.2.3 Context Storage . 23 2.2.4 Packet Resequencing . 24 2.2.5 Overlapping Retransmissions . 26 2.2.6 Idle Flows . 26 v 2.2.7 Resource Exhaustion . 27 2.2.8 Selective Flow Monitoring . 28 2.2.9 Multi-Node Monitor Coordination . 29 2.2.10 Fragmentation . 29 2.2.11 Flow Modification . 31 2.2.12 Bi-Directional Traffic Monitoring . 32 3 Related Work :::::::::::::::::::::::::::::::::: 34 3.1 Network Monitoring Systems . 34 3.2 Software-Based Network Monitors . 35 3.3 Hardware-Based Network Monitors . 37 3.4 Packet Classification . 38 3.5 Related Technologies . 41 3.5.1 Load Balancers . 41 3.5.2 SSL Accelerators . 43 3.5.3 Intrusion Detection Systems . 44 3.5.4 TCP Offload Engines . 46 3.6 Hardware-Accelerated Content Scanners . 48 3.7 Summary . 51 4 Architecture ::::::::::::::::::::::::::::::::::: 52 4.1 Initial Investigations . 52 4.1.1 TCP-Splitter . 53 4.1.2 StreamCapture . 55 4.2 TCP-Processor . 57 4.3 Application Interface . 61 4.4 Extensibility . 63 4.5 Multiple FPGA Coordination . 64 5 Environment :::::::::::::::::::::::::::::::::: 67 5.1 Field-programmable Port Extender . 67 5.2 Washington University Gigabit Switch . 68 5.3 NCHARGE . 69 5.4 FPX-in-a-Box . 70 5.5 Protocol Wrappers . 71 vi 6 TCP-Processor Internals :::::::::::::::::::::::::::: 73 6.1 Endianness . 74 6.2 Packet Parameters . 75 6.3 Flow Control . 75 6.4 External Memories . 76 6.5 Configuration Parameters . 76 6.6 TCP Processor . 78 6.7 TCP Proc . 78 6.8 TCP Input Buffer . 80 6.9 TCP Engine . 82 6.10 State Store Manager . 88 6.11 TCP Routing . 93 6.11.1 Client Interface . 95 6.12 TCP Egress . 96 6.13 TCP Stats . 100 7 StreamExtract Circuit ::::::::::::::::::::::::::::: 104 7.1 StreamExtract Module . 105 7.2 StreamExtract . 106 7.3 LEDs . 106 7.4 Serialization/Deserialization (Endoding/Decoding) . 106 7.4.1 TCPSerializeEncode . 109 7.4.2 TCPSerializeDecode . 111 7.5 Implementation . 112 8 TCP-Lite Wrappers :::::::::::::::::::::::::::::: 114 8.1 TCPDeserialize . 114 8.2 TCPReserialize . 115 8.3 PortTracker . 115 8.3.1 PortTracker Module . 116 8.3.2 PortTrackerApp . 117 8.3.3 ControlProcessor . 117 8.4 Scan . 119 8.4.1 Scan Module . 120 8.4.2 ScanApp . 121 8.4.3 ControlProcessor . 126 vii 8.4.4 StateStore . 129 9 Analysis ::::::::::::::::::::::::::::::::::::: 131 9.1 Test Setup . 131 9.2 Data Collection . 132 9.3 Results . 133 9.3.1 Denial of Service Attack . 135 9.3.2 Virus Detection . 137 9.3.3 Spam . 138 9.3.4 Traffic Trends . 139 9.3.5 TCP Flow Classification . 141 9.3.6 Traffic Types . 143 10.