Optimizing Xcast Treemap Performance with NFV and SDN T. Khoa Phan Joined work with David Griffin and Miguel Rio University College London Next Generation Networking workshop, July 2016 Facebook Livestream System A B source Origin server C Edge cache Live stream server Origin server D E Fig. 4: Facebook livestream system - CDN [1] • 98% of user requests can be served immediately by edge caches • Each edge cache can serve up to 200,000 users simultaneously [1] https://code.facebook.com/posts/1653074404941839/under-the-hood-broadcasting-live-video-to-millions/ 2 What is Xcast Treemap? S Breath-first tree traversal A B C D E List of IP addresses A 2 0 2 0 0 Treemap Sender S creates packets: B C A B C D E Src: S, Dest: A Payload 2 0 2 0 0 D E Unicast part Xcast treemap part (optional) Today router only understands unicast part. Xcast router lookups and forwards for each IP in the list. Xcast end-host and Xcast router software are available (in IPv6): http://www.ee.ucl.ac.uk/~uceetkp/Xcast_software.zip 3 How Xcast Treemap works? Unicast routing table at R3 Dest Next hop Full Xcast packet header created by S: C C A B C D E D D Src: S, Dest: A Payload 2 0 2 0 0 E E Dest in List dests in unicast part X6Tpart A B D S A | A B C D E B | B D | D A S C B C R1 R2 C | C D E R3 C | C D E E | E Dest Next hop D E E A A Today router B, C, D, E R2 Unicast routing table at R1 Fig. 1: only today routers 4 How Xcast Treemap works? Unicast routing table at R3 Dest Next hop Full Xcast packet header created by S: C C A B C D E D D Src: S, Dest: A Payload 2 0 2 0 0 E E A B D S A | A B C D E B | B D | D A C | C S C B C R1 R C | C D E R3 C | C D E 2 E | E Dest Next hop D E E A A Xcast router B, C, D, E R2 Today router Unicast routing table at R1 Fig. 2: mixture of Xcast and today routers 5 Facebook Livestream System A B source Origin server C Edge cache Live stream server Origin server D E Fig. 4: Facebook livestream system - CDN [1] • 98% of user requests can be served immediately by edge caches • Each edge cache can serve up to 200,000 users simultaneously [1] https://code.facebook.com/posts/1653074404941839/under-the-hood-broadcasting-live-video-to-millions/ 6 Facebook Livestream System C E source Origin server A Edge cache Live stream server A B C D E D 2 0 2 0 0 Origin server B Fig. 5: Xcast with Facebook live stream • The edge cache uses only 1 slot instead of 5 slots • In this example, each edge cache can serve up to 1,000,000 users 7 Facebook Livestream System E D C Edge Edge cache A cache A C leaves session A B D E A B C D E D 2 1 0 0 2 0 2 0 0 B E B Xcast Treemap is a stateless protocol. The overlay tree can be changed by the edge cache in silence!!! A B D A | A B C D E D | D B | B C C | C NFV and SDN will help to deploy R1 R2 C | C D E X3 E | E C | C D E Xcast routers. E Nothing happens to D and E when C leaves the session because Xcast router X3 forwards data to them 8 Xcast Treemap with NFV • Integrate Xcast router service in lightweight containers (Docker and Kubenetes) (on-going work) • Users can deploy their own Xcast gateways, no touch the core network • Smart placement (on demand) for Xcast routers (on-going work) S Core network Gateway Xcast Xcast gateway gateway A B C D 9 Xcast Treemap with SDN • Use SDN to separate and control Xcast traffic from other traffic (on-going work) • Traffic engineering for Xcast traffic (on-going work) • SDN controller helps to build a good overlay tree (on-going work) 10 Xcast Treemap with SDN R2 R1 S S X3 A A B C B C Shortest path routing SDN controller R2 R1 S X3 A B C Traffic engineering for Xcast traffic 11 Conclusions • Xcast Treemap vs. P2P: - Quick adaptation when end-hosts join/leave. - Significantly reduce latency and save bandwidth if we can deploy Xcast routers/gateways. • Advantages of Xcast Treemap vs. IP multicast: - Xcast cost is as low as unicast (no need routers upgraded, no multicast protocol, resilience with network failure, stateless protocol). - No problem with inter-domain multicast as Xcast works like unicast and P2P. • On-going work: - NFV to ease Xcast router deployment - SDN to manage and to optimize traffic engineering for Xcast traffic Xcast end-host and Xcast router software are available (in IPv6): http://www.ee.ucl.ac.uk/~uceetkp/Xcast_software.zip 12 Backup slides 13 A bit of Xcast history • Xcast concept: L. Aguilar “Datagram Routing for Internet Multicasting”, SIGCOMM, 1984. • IP Multicast: S. Deering “Host extensions for IP multicasting”, RFC 1112, 1989. - Internet community preferred IP multicast over Xcast concept. • Xcast was back: R. Boivie, et al. “Explicit Multicast (Xcast) Concepts and Options”, RFC 5058, 2007. - No overlay tree, only work for small multicast group - Need Xcast router • Our work: Xcast6 Treemap - Support overlay multicast and work for large multicast group - Xcast routers are optional 14 How Xcast work with large group? Apply similar approach as how we scale the Facebook edge cache server if the sender has very high capacity, otherwise partition the overlay tree and send data as follows: F G H I J 4 0 0 0 0 D E J I By selecting some end- A B C D E hosts (A and C in this 3 0 0 1 0 Figure) to behave like A C F the original sender. By this way, each packet header contains only 5 B H IP addresses (but can K L M O N N support thousands of 1 1 2 0 0 G K users like P2P). O 15 L M How Xcast Treeamap works? S A B C Dest in List dests in unicast part X6Tpart A B D D E A | A B C D E D | D A | A S C R1 R2 C | C D E R3 B | B C D E E | E E Today router Xcast router B has no child in the overlay tree but B sees [C D E] when receiving packets. It means that the overlay tree is broken by Xcast routers located somewhere on the network. Therefore B will forward the packets to C (the first in the list [C D E]) 16 How Xcast works? S A B C Dest in List dests in unicast part X6Tpart A B D D E A | A B C D E B | B D | D S C R1 R2 C | C D E R3 C | C D E E | E E Today router Xcast router 17 How Xcast works? Full Xcast packet header created by S: A B C D E Src: S, Dest: A Payload 2 0 2 0 0 List dests in Dest in unicast part Xcast part A B D A | A B C D E A | A B | B S C R1 B | B C D E R2 C | C D E R3 Dest Next hop E A A Xcast router B, C, D, E R2 Unicast routing table at R1 Fig. 3: all routers are Xcast routers 18 Facebook Livestream System E D C Edge A Edge cache cache A C leaves session A B D E A B C D E D 2 1 0 0 2 0 2 0 0 B E B Edge caches can change the pkt header in silence!!! New viewers join: the edge caches will serve them directly as how the system is working now, and will move them to an overlay tree when they are in the session long enough (stable viewers). A B D A | A B C D E B | B D | D C | C C R R C | C D E X 1 C | C D E 2 3 E | E Nothing happens to D and E when C leaves the session E 19 because Xcast router X3 forwards data to them .