GENI Engineering Conference March 4, 2008 A couple of platforms (Or: “Why can’t I innovate in my wiring closet?”) Nick McKeown [email protected] The Stanford Clean Slate Program http://cleanslate.stanford.edu GENI: How to try new network architectures in my local network? PlanetLab Suitable for distributed systems using overlays Not suitable for radical network architectures Performance is limited XORP Open-source router code for Linux, FreeBSD, etc. Performance is limited Point solution OpenWRT Linux for wireless access points and routers VINI, WUSTL SPP, … Designed for large nationwide backbone infrastructure The Stanford Clean Slate Program http://cleanslate.stanford.edu Outline NetFPGA ¾ Open source networking hardware ¾ 4-ports of 1GE ¾ Operates at line-rate ¾ Available for $500 (university) OpenFlow ¾ Enabling innovation on campus ¾ Standard way to control flow-tables in commercial switches and routers ¾ Being deployed at Stanford ¾ Consider deploying it at your campus too The Stanford Clean Slate Program http://cleanslate.stanford.edu Open source networking hardware CPU Memory PCI PC with NetFPGA 1GE FPGA 1GE 1GE Memory 1GE Rack of 1U servers NetFPGA Board 96 x 1GE ports 9 Hardware available from 3rd party ~$500 (universities) 9 PCI board, or pre-built system (desktop or rack-mount) The Stanford Clean Slate Program http://cleanslate.stanford.edu NetFPGA: Usage Models Types of user 1. Teachers 2. Students 3. Researchers 4. Commercial development Supported Use Models 1. Router Kit 2. Supported Modular Reference Designs 3. Free Design (or 3rd party design) All software and designs available under BSD-like license The Stanford Clean Slate Program http://cleanslate.stanford.edu 1 # ge sa U NetFPGA “Router Kit” User-space development, 4x1GE line-rate forwarding OSPF BGP CPU Memory My Protocol user kernel Routing TablePCI “Mirror” Fwding Packet 1GE Table Buffer FPGA 1GE IPv4 1GE Router Memory 1GE The Stanford Clean Slate Program http://cleanslate.stanford.edu 2 # ge sa U NetFPGA “Reference Router” Modules in hardware PW-OSPF Verilog CPU Memory Java GUI EDA Tools Front Panel (Xilinx, (Extensible) Mentor, etc.) PCI NetFPGA Driver 1.1. DesignDesign 2.2. SimulateSimulate 1GE 1GE L3 L2 In3.3. QSynthesizeSynthesize 4.4. DownloadDownload FPGA 1GE Parse Parse Mgmt 1GE 1GE IP My Out Q 1GE Memory Lookup Block Mgmt 1GE 1GE Verilog modules interconnected by FIFO interfaces The Stanford Clean Slate Program http://cleanslate.stanford.edu 3 # ge sa U NetFPGA Free Design Verilog CPU Memory EDA Tools (Xilinx, Mentor, etc.) 1.1. DesignDesign PCI NetFPGA2.2. DriverSimulateSimulate 3.3. SynthesizeSynthesize 4.4. DownloadDownload 1GE 1GE FPGA 1GE My Design 1GE 1GE 1GE Memory (1GE MAC is soft/replaceable) 1GE 1GE The Stanford Clean Slate Program http://cleanslate.stanford.edu Resources http://NetFPGA.org Available ¾ NetFPGA hardware ¾ Linux Driver ¾ Java GUI ¾ Test & regression software Courseware ¾ Undergraduate EE: Ethernet switch ¾ Masters EE/CS: IPv4 router hardware and software Tutorials ¾ 6 tutorials worldwide in 2008 ¾ 1 week Stanford summer camp The Stanford Clean Slate Program http://cleanslate.stanford.edu Resources (2) http://NetFPGA.org Reference Designs ¾ 4x1GE NIC ¾ 4x1GE IPv4 Router ¾ 4x1GE OpenFlow (soon) User-contributed ¾ NAT ¾ IEEE 1588 Timing Sync ¾ Real-time buffer monitoring ¾ RCP (Rate Control Protocol) The Stanford Clean Slate Program http://cleanslate.stanford.edu Funding http://NetFPGA.org Launched with funds from NSF EIA Program The Stanford Clean Slate Program http://cleanslate.stanford.edu Outline NetFPGA – Open source networking hardware – 4-ports of 1GE – Operates at line-rate – Available for $500 (university) OpenFlow – Enabling innovation on campus – Standard way to control flow-tables in commercial switches and routers – Being deployed at Stanford – Consider deploying it at your campus too The Stanford Clean Slate Program http://cleanslate.stanford.edu Innovations in campus wiring closets Experiments we’d like to do ¾ Mobility management ¾ Network-wide energy management ¾ New naming/addressing schemes ¾ Network access control Problem with our network ¾ Paths are fixed (by the network) ¾ IP-only ¾ Addresses dictated by DNS, DHCP, etc ¾ No means to add our own processing The Stanford Clean Slate Program http://cleanslate.stanford.edu OpenFlow Switching 1. A way to run experiments in the networks we use everyday. 2. Bring GENI to college campuses. A “pragmatic” compromise Allow researchers to run experiments in their network… …without requiring vendors to expose internal workings. Basics An Ethernet switch (e.g. 128-ports of 1GE) An open protocol to remotely add/remove flow entries The Stanford Clean Slate Program http://cleanslate.stanford.edu Experimenter’s Dream (Vendor’s Nightmare) User- StandardStandard User- Experimenter writes sw defined NetworkNetwork defined experimental code Processing hw ProcessingProcessing Processing on switch/router The Stanford Clean Slate Program http://cleanslate.stanford.edu No obvious way Commercial vendor won’t open software and hardware development environment ¾ Complexity of support ¾ Market protection and barrier to entry Hard to build my own ¾ Prototypes are flakey ¾ Software only: Too slow ¾ Hardware/software: Fanout too small (need >100 ports for wiring closet) The Stanford Clean Slate Program http://cleanslate.stanford.edu Furthermore, we want… Isolation: Regular production traffic untouched Virtualized and programmable: Different flows processed in different ways Equipment we can trust in our wiring closet Open development environment for all researchers (e.g. Linux, Verilog, etc). Flexible definitions of a flow ¾ Individual application traffic ¾ Aggregated flows ¾ Alternatives to IP running side-by-side ¾ … The Stanford Clean Slate Program http://cleanslate.stanford.edu OpenFlow Switching Controller OpenFlow Switch specification ow OpenFlow Switch enFl PC Op ol toc Pro SSL sw SecureSecure ChannelChannel hw FlowFlow TableTable The Stanford Clean Slate Program http://cleanslate.stanford.edu Flow Table Entry “Type 0” OpenFlow Switch Rule Action Stats Packet + byte counters 1. Forward packet to port(s) 2. Encapsulate and forward to controller 3. Drop packet 4. Send to normal processing pipeline Switch MAC MAC Eth VLAN IP IP IP TCP TCP Port src dst type ID Src Dst Prot sport dport + mask The Stanford Clean Slate Program http://cleanslate.stanford.edu OpenFlow “Type 1” Definition in progress Additional actions ¾Rewrite headers ¾Map to queue/class ¾Encrypt More flexible header ¾Allow arbitrary matching of first few bytes Support multiple controllers ¾Load-balancing and reliability The Stanford Clean Slate Program http://cleanslate.stanford.edu Secure Channel SSL Connection, site-specific key Controller discovery protocol Encapsulate packets for controller Send link/port state to controller The Stanford Clean Slate Program http://cleanslate.stanford.edu Server room OpenFlow OpenFlow Access Point Controller PC OpenFlow-enabled Commercial Switch OpenFlow OpenFlow Normal SecureSecure Software ChannelChannel Normal FlowFlow Datapath TableTable The Stanford Clean Slate Program http://cleanslate.stanford.edu OpenFlow Usage Models 1. Experiments at the flow level User-defined routing protocols Admission control • Experiment-specific controllers Network access control • Static or dynamic flow-entries Network management Energy management VOIP mobility and handoff … 2. Experiments at the packet level Slow: Controller handles packet processing Fast: Redirect flows through programmable hardware Modified routers, firewalls, NAT, congestion control… 3. Alternatives to IP The Stanford Clean Slate Program http://cleanslate.stanford.edu Example Experiment at the flow level Mobility Lots of interesting questions • Management of flows • Control of switches • Access control of users and devices • Tracking user location and motion The Stanford Clean Slate Program http://cleanslate.stanford.edu Experiments at the packet level Controller PC OpenFlow-enabled Commercial Switch Normal SecureSecure Software ChannelChannel Normal FlowFlow Datapath TableTable Laboratory NetFPGA The Stanford Clean Slate Program http://cleanslate.stanford.edu OpenFlow Usage Models 1. Experiments at the flow level 2. Experiments at the packet level 3. Alternatives to IP Flow-table is Layer-2 based e.g. new naming and addressing schemes … The Stanford Clean Slate Program http://cleanslate.stanford.edu OpenFlow Consortium http://OpenFlowSwitch.org Goal: Evangelize OpenFlow to vendors Free membership for all researchers Whitepaper, OpenFlow Switch Specification, Reference Designs Licensing: Free for research and commercial use The Stanford Clean Slate Program http://cleanslate.stanford.edu OpenFlow: Status Commercial Ethernet switches and routers ¾ Working with six vendors to add to existing products ¾ Expect OpenFlow “Type 0” to be available in 2008-09 Reference switches ¾ Software: Linux and OpenWRT (for access points) ¾ Hardware: NetFPGA (line-rate 1GE; available soon) ¾ Working on low-cost 48-port 1GE switch based on Broadcom reference design Reference controller ¾ Simple test controller ¾ NOX controller (Martin Casado; available soon) The Stanford Clean Slate Program http://cleanslate.stanford.edu Deployment at Stanford Stanford Computer Science Department Gates Building ~1,000 network users 23 wiring closets Stanford Center for Integrated Systems (EE) Paul Allen Building ~200 network users 6 wiring closets Working with HP Labs and Cisco on deployment The Stanford Clean Slate Program http://cleanslate.stanford.edu If you are interested in deploying OpenFlow on your campus… Please contact me! [email protected] http://OpenFlowSwitch.org.