Open vSwitch Ben Pfaff VMware What is Open vSwitch? Semi-official description: Open vSwitch is a production quality, multilayer virtual switch licensed under the open source Apache 2.0 license. It is designed to enable massive network automation through programmatic extension, while still supporting standard management interfaces and protocols (e.g. NetFlow, sFlow, SPAN, RSPAN, CLI, LACP, 802.1ag). The exciting parts: ● Write a program to control your network. ● Fast! (and getting faster) ● Portable (with new ports coming) Where is Open vSwitch Used? ● Broad support: – Linux, FreeBSD, NetBSD, Windows, ESX – KVM, Xen, Docker, VirtualBox, Hyper-V, … – mininet, OpenStack, CloudStack, OpenNebula, … ● Widely used: – Most popular OpenStack networking backend – Default network stack in XenServer – 2,180 hits in Google Scholar – Thousands of subscribers to OVS mailing lists ● Hundreds of contributors (Incomplete) List of Contributors Programming Model OpenFlow controller Ingress Flow Table Flow Table … Flow Table Egress Port 0 1 N Port(s) Flow Tables IF a AND b AND ... THEN <actions1> ELSE IF c AND d AND … THEN <actions2> ELSE … Conditions Actions Packet header matches Output to port Metadata matches: Modify packet header or metadata ingress port Add/remove VLAN, MPLS, etc. queuing information Go to or recurse into flow table tunnel information Push/pop stack Bitwise matches Modify flow table Range matches* “Drop” Set membership matches** ... ... priority=55, in_port=1,tcp,tcp_src=80, actions=output:2 Programming Model Properties ● Somewhat good match for hardware ASICs ● Natural for network admins, unnatural for programmers ● Turing complete (with OVS extensions) ● Easy to implement, hard to optimize – (Wasn't obvious it needed optimization.) ● Embarrassingly parallel* OVS Programming Example ● Notes and test cases in source tree ● The curse of “normal”. ● Normal pipeline: 0. Admission control (reserved multicast, …). 1. Ingress VLAN processing 2. Learn source MAC+VLAN on ingress port. 3. Look up egress port for dest MAC+VLAN. 4. Egress VLAN processing. Example: Test Case trunk port p1 br0 p2 p3 p4 VLAN 20 VLAN 30 access ports Stage 0: Admission Control priority=32768, dl_src=01:00:00:00:00:00/01:00:00:00:00:00, actions=drop priority=32768, dl_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0, actions=drop priority=0, actions=resubmit(,1) Stage 1: Ingress VLAN Processing priority=0, actions=drop trunk port p1 priority=99, in_port=1, br0 actions=resubmit(,2) p3 p2 p4 VLAN 20 VLAN 30 priority=99, in_port=2, vlan_tci=0, access ports actions=mod_vlan_vid:20, resubmit(,2) priority=99, in_port=3, vlan_tci=0, actions=mod_vlan_vid:30, resubmit(,2) priority=99, in_port=4, vlan_tci=0, actions=mod_vlan_vid:30, resubmit(,2) Stage 2: VLAN+MAC learning actions= learn( table=10, NXM_OF_VLAN_TCI[0..11], NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[], load:NXM_OF_IN_PORT[] ->NXM_NX_REG0[0..15]), resubmit(,3) Stage 3: Look up egress port priority=50, actions=resubmit(,10), resubmit(,4) priority=99, dl_dst=01:00:00:00:00:00/01:00:00:00:00:00, actions=resubmit(,4) Stage 4: Egress VLAN Processing trunk port reg0=1, actions=1 p1 br0 p3 p2 p4 reg0=2, actions=strip_vlan,2 VLAN 20 VLAN 30 reg0=3, actions=strip_vlan,3 access ports reg0=4, actions=strip_vlan,4 reg0=0, priority=99, dl_vlan=20, actions=1,strip_vlan,2 reg0=0, priority=99, dl_vlan=30, actions=1,strip_vlan,3,4 reg0=0, priority=50, actions=1 priority=0, actions=drop Lessons Learned ● The best real packet classifiers are not found in papers. ● "I've rewritten the *!@# datapath N times, I'm not going to let iteration N+1 stop me." ● Standardization does not fix warts. ● Reactive programming is seductive, but doesn't scale. ● In-band control is hard. ● Guessing about performance is risky. OVN: The Open Virtual Network for Open vSwitch The Case for Network Virtualization • Network provisioning needs to be self-service. • Virtual network needs to be abstracted from physical. • Virtual network needs same features as physical. A B D E H H S S V V 1 2 A B C F C R S D E G H S S F G H L Cl e o g u a d cy P P h h ys ys ic ic al al N N et et w w or or k k Why Start OVN? • Virtual networking for OVS, done from scratch using experience built up in the OVS project • Natural evolution of the project--OVS had only provided the components to build a distributed switch • Cloud management systems agnostic What is OVN? • Open source L2/L3 network virtualizaton for Open vSwitch (OVS): –Logical switches –L2/L3/L4 ACLs (Security Groups) – Logical routers – Multple tunnel overlays (Geneve, STT, and VXLAN) – TOR-based and sofware-based logical-physical gateways • Works on same platorms as OVS: –Linux (KVM and Xen) – Containers (Docker) – DPDK – Hyper-V • Integraton with: –OpenStack Neutron –Docker –Other CMSes The Particulars • Developed by the same community as Open vSwitch • Vendor-neutral • Design and implementaton all occur in public • Developed under the Apache license Goals • Producton-quality • Straightorward design • Scale to 1000s of hypervisors (each with many VMs/containers) • Improved performance and stability over existng OpenStack OVS plugin O p e n St a OVNck Architecture / o C M v S nPlN - uo n gir o nt rt h b h o S d u o n u o d t o v h D b v n B o … n - u - o c n oc v o o d ov o s v n ns v d s- D tr trd s- b v B ol olb v - s le le- s s w r sr w e it e it r c r c v h v h e d H e d r r H V V - - 1 n Architecture • Configuration coordinated through databases •Logical flows, don’t worry about physical topology • Local controller converts logical flow state into physical flow state • Desired state clearly separated from run-time state •Based on the architecture we wanted based on seeing a number of others using OVS Thank you! Ben Pfaff [email protected] Feel free to email me with questions or if you are interested in a job or internship at VMware..