Peer-to-peer Virtual Private Networks and Applications
Renato Jansen Figueiredo Associate Professor Cloud and Autonomic Computing Center/ACIS Lab University of Florida Visiting Researcher at VU Backdrop
Virtual machines in cloud computing On-demand, pay-per-use, user-configurable Federated environments End-to-end Internet connectivity hindered by address space and presence of NATs, firewalls Network virtualization – seamlessly connecting virtual machines across multiple providers Applications Distributed virtual private clusters Education, training in distributed computing Private communication among Internet users extending online social networks
2 Rationale
Virtualization techniques for decoupling, isolation, multiplexing also apply to networking E.g. VLANs, VPNs However, there are challenges in configuration, deployment, and management Peer-to-peer techniques provide a basis for scalable routing, and self-management Software routers, integration at network end-points enables deployment over existing infrastructure Architecture, design needs to account for connectivity constraints, and support TCP/IP efficiently; optimize for common cases
3
Application Examples
Cloud-bursting Run additional worker VMs on a cloud provider Extending enterprise LAN to cloud VMs – seamless scheduling, data transfers Federated “Inter-cloud” environments Multiple private clouds across various institutions Virtual machines can be deployed on different sites and form a distributed virtual private cluster Hands-on laboratories for classes Deploy your own cluster Connecting devices of social network peers Media streaming, file sharing, gaming, …
4 Talk - Outlook
Background Architecting self-organizing virtual networks Topology, routing, tunneling, addressing, NAT traversal, performance Applications in Grid/cloud and end-user environments Virtual Private Clusters Social VPNs Applications in education FutureGrid
5 Resource Virtualization
Virtual machines (Xen, VMware, KVM) paved the way to Infrastructure-as-a-Service (IaaS) Computing environment decoupled from physical infrastructure Pay-as-you-go for computing cycles
Virtual networks complement virtual machines for increased flexibility and isolation in IaaS VMs must communicate seamlessly – regardless of where they are provisioned Traffic isolation; security, resource control
6 Virtual Machines and Networks
Virtual V2 Infrastructure
V3 V1
VMM + VN
Physical Infrastructure Domain B
WAN Domain A Domain C
7 Virtual Networks
Single infrastructure, many virtual networks E.g. one per user, application, project, social network… Each isolated and independently configured Addressing, protocols; authentication, encryption Multiplexing physical network resources Network interfaces, links, switches, routers
8 Network Virtualization – Where?
Virtualized endpoints Software Software
Network Network Fabric Network Device Device
(Virtual) (Virtual) machine machine
Virtualized Fabric (e.g VLAN, OpenSwitch)
9 Landscape
Peer-wise Internet connectivity constrained IPv4 address space limitations; NATs, firewalls Challenges - shared environment Lack of control of networking resources Cannot program routers, switches Public networks – privacy is important Often, lack privileged access to underlying resources May be “root” within a VM, but lacking hypervisor privileges Dynamic creation, configuration and tear-down Complexity of management
10 Peer-to-Peer Virtual Networks
Overview User-level IP overlays deployable on Internet end resources (software routers, virtual NICs) Why virtual? Hide complexities associated with NAT traversal, IPv4 address space constraints from applications Support unmodified applications Why peer-to-peer? Self-organizing - reduce management complexity and cost Decentralized architecture for scalability and robustness
11 The IP-over-P2P (IPOP) Approach
Isolation Virtual address space decoupled from Internet Packets picked, encapsulated, tunneled and delivered within the scope of virtual network Self-organization Overlay topology, routing tables Autonomously deals with joins, leaves, failures Decentralized P2P messaging architecture No global state, no central point of failure Tunnels (UDP, TCP, …), routing Decentralized NAT traversal No need for STUN server infrastructure
[IPDPS 2006, Ganguly et al]
12 IPOP: Architecture Overview
Unmodified applications Capture/tunnel, scalable, Connect(10.10.1.2,80) resilient, self-configuring Routing; object store Application Virtual Router
VNIC 10.10.1.1
Wide-area Virtual Application Overlay network Router Isolated, private virtual VNIC address space 10.10.1.2 P2P Overlay (Brunet)
Bi-directional ring ordered by 160-bit IPOPid’s Structured connections: • “Near”: with neighbors n4 IPOPid • “Far”: across the ring n3 n5 n2 n6
Multi-hop path n1 n7 between n1 and n7
Far n12 n8 Near n11 n9 n1 < n2 < n3 <……. < n13 < n14 n10 Overlay: Edges and Routing
Overlay edges Multiple transports: UDP, TCP, TLS NAT traversal (UDP hole-punching) Greedy routing Deliver to peer closest to destination IPOPid Constant # of edges per node (average k) O((1/k)log2(n)) overlay hops On-demand edges Created/trimmed down based on IP communication
15 Creating Overlay Edges
Overlay path
CTM request CTM reply Link request A B A’s endpoint URIs: B’s endpoint URIs: tcp://10.5.1.1:3000 tcp://172.16.2.1:5000 (local) udp://129.15.56.9:6000 udp://128.227.56.9:4433 (NAT – learned) • A sends a Connect-to-me (CTM) request to B’s IPOPid • Contains all its URIs (UDP/TCP IP:port endpoints) • Routed over P2P overlay to B • B sends CTM reply with its URIs – overlay routed • B initiates “linking” with A • Attempts linking with parallel requests to A’s URIs 16 NAT Traversal
Direct edge between A and B
A B
Technique for “cone” UDP NATs:
A’s link request message to B creates ephemeral state in A’s NAT allowing messages from B to pass through NAT (and vice-versa)
Overlay: manage keep-alives so NAT mapping “holes” stay open; re-link if NAT mappings expire 17 Naming and Multiplexing
One P2P overlay can multiplex multiple VNs E.g. multiple virtual clusters from different projects IP routing within the scope of a namespace
User-provided string identifies IPOP namespace Each IPOP node is configured with a namespace
IP-to-P2P address resolution: DHT-Get(namespace:IP) -> IPOPid
18 Managing Virtual IP Addresses
Address assignment: static, or dynamic Supports DHCP Store configuration (including base address, mask) on DHT entry bound to namespace
DHCP proxy runs on each IPOP node Pick DHCP request Lookup DHCP configuration for namespace Guess an IP address at random within range Attempt to store in DHT; wait for majority to acknowledge; retry upon failure
19 Optimization: On-demand edges
At each node:
Count IP-over-P2P packets to other nodes
When number of packets within an interval exceeds threshold:
Initiate connection setup; create edge
Trimming on-demand edges no longer in use
Overhead involved in connection maintenance
20 Optimization: Tunnel Edges
Peers X, Y may not be able to communicate directly if they are behind symmetric NATs
X, Y exchange list of neighbor URIs Each attempts to create edge to common intermediary Z to serve as proxy
Routing – abstracted as regular overlay edge X-Y connected by virtual edge Useful to maintain ring topology in the face of failures (routing outages, symmetric NATs)
21 Implementation
IPOP open-source system C# user-level router Tap virtual network device Performance 1GbE physical LAN
Latency (ms) Bwidth (Mb/s) Mem (KB) Host 0.27 941 n/a IPOP 0.52 284 38312 IPOP+sec 0.75 55 50976
22 Performance (WAN)
EC2/UF EC2GoGrid UF/GoGrid Netperf stream 89.2 35.9 30.2 native (Mbps) Netperf stream 75.3 19.2 25.7 IPOP (Mbps) Netperf RR 13.4 11.1 10.0 trans/s native Netperf RR 13.3 10.7 9.8 trans/s IPOP Related Work
There exist several VPN technologies: Enterprise VPNs (e.g. Cisco); Open-source (e.g OpenVPN); Consumer/gaming/SMB (e.g. Hamachi) Not easily applicable to federating cloud resources Proprietary code; difficulty in configuration/management Research work in the context of Grid/cloud computing VNET (Northwestern University), VIOLIN (Purdue University), Private Virtual Cluster (INRIA), ViNe (Tsugawa, Fortes @ UF) Smartsockets @ VU
24 IPOP Social Networks
Users now commonly manage relationships to social peers through Online Social Networks Facebook, Google+ Communication hindered by OSN provider APIs, privacy concerns A generic IP network can enable existing and new social network applications But users don’t have public IPs, don’t want to necessarily open NATs/firewalls to all users Users don’t want to configure and discover network services manually
25 Social VPNs
Alice's Compute Node Alice's Friend's Compute Node Bob's Compute Node on EC2
IP-over-P2P Tunnel OSN
XMPP
Alice Bob Carl Social VPNs
From a user’s perspective: it’s simple My computer gets a virtual network card It connects me directly to my social peers All IP packets: authenticated, encrypted, end-to-end Leverage well-known PKI techniques No configuration besides establishing social links All I need to do to is log in to a web based social network Applications, middleware work as if the computers were on the same local-area network Including multicast-based resource discovery – UPnP, mDNS
27 Applications
Social VPN is not the application It is not tied to an application either It enables applications that are of interest for collaboration Security – needed beyond network layer Authenticated end-to-end private IP tunnels provide a foundation Traditional applications Media streaming, desktop sharing, file sharing, cycle sharing Platform for decentralized social network applications Fault-tolerant micro-blogging, private file sharing, ..
28 Social VPN Internals
IPOP NAT traversal and routing core Private end-to-end tunnels Peer discovery and certificate exchange XMPP – Jabber, Google Facebook APIs (was in first prototype; no longer in the code) Dynamic IP address assignment Facebook: more users than IPv4 24-bit private space Also must avoid conflicts with local private networks, and support mobility
29 Addressing and Mapping
160-bit P2P IDs used for overlay routing Each node generates random P2P ID Node issues a self-signed public key certificate with its P2P identifier; publishes through OSN APIs Certificates of friends’ nodes are discovered, retrieved, revoked through OSN APIs IPv4 addresses seen by applications Dynamically-generated non-conflicting private subnet Local node and friends’ nodes are mapped dynamically to addresses within range Naming possible through SocialDNS IP src/dest addresses translated (ports are not) [COPS 2008]
Address Translation
Alice's Compute Node Alice's Friend's Compute Node Bob's Compute Node on EC2
Send-to Alice BobP2P Recv-from 0.1 2.2 AliceP2P 3.4 0.1
SVPN: 192.168.0.0/16 SVPN: 172.16.0.0/16 Alice: 192.168.0.1 Bob: 172.16.0.1 Bob: 192.168.2.2 -> BobP2P Alice: 172.16.3.4 -> AliceP2P Group-oriented Social VPNs
SocialVPN focuses on unstructured peer-to- peer communication User’s VPN scoped by their own social links Powerful abstraction, but there are applications require all-to-all connectivity E.g. a virtual private cluster How to use relationships established through Web-based portals to create virtual clusters? Group-oriented VPNs Software packaging in virtual appliance images
32 Use case: Education and Training
Importance of experimental work in systems research Needs also to be addressed in education Complement to fundamental theory FutureGrid: a testbed for experimentation and collaboration Education and training contributions: Lower barrier to entry – pre-configured environments, zero- configuration technologies Community/repository of hands-on executable environments: develop once, share and reuse
33 Educational appliances in FutureGrid
A flexible, extensible platform for hands- on, lab-oriented education on FutureGrid Executable modules – virtual appliances Deployable on FutureGrid resources Deployable on other cloud platforms, as well as virtualized desktops Community sharing – Web 2.0 portal, appliance image repositories An aggregation hub for executable modules and documentation
34 What is a virtual appliance?
A virtual appliance packages software and configuration needed for a particular purpose into a virtual machine “image”
The virtual appliance has no hardware – just software and configuration
The image is a (big) file
It can be instantiated on hardware
35 Virtual appliance example
Linux + Apache + MySQL + PHP A web server Another Web server LAMP image
instantiate Virtualization copy Layer
Repeat…
36 Clustered applications
Replace LAMP with the middleware of your choice – e.g. MPI, Hadoop, Condor MPI image An MPI worker Another MPI worker
instantiate Virtualization copy Layer
Repeat…
37 Grid appliance - virtual clusters
Same image, per-group VPNs
Hadoop Group + VPN Virtual Network A Hadoop worker Another Hadoop worker instantiate
copy Virtual GroupVPN machine Credentials Repeat… (from Web site) Virtual IP - DHCP Virtual IP - DHCP 10.10.1.1 10.10.1.2
38 Grid appliance clusters
Virtual appliances Encapsulate software environment in image Virtual disk file(s) and virtual hardware configuration The Grid appliance Encapsulates cluster software environments Current examples: Condor, MPI, Hadoop Homogeneous images at each node Virtual Network connecting nodes forms a cluster Deploy within or across domains
39 Grid appliance internals
Host O/S Linux Grid/cloud stack MPI, Hadoop, Condor, … Glue logic for zero-configuration Automatic DHCP address assignment Multicast DNS (Bonjour, Avahi) resource discovery Shared data store - Distributed Hash Table Interaction with VM/cloud
40 One appliance, multiple hosts
Allow same logical cluster environment to instantiate on a variety of platforms Local desktop, clusters; FutureGrid; Amazon EC2; Science Clouds… Avoid dependence on host environment Make minimum assumptions about VM and provisioning software Desktop: 1 image, VMware, VirtualBox, KVM Para-virtualized VMs (e.g. Xen) and cloud stacks – need to deal with idiosyncrasies Minimum assumptions about networking Private, NATed Ethernet virtual network interface
41 Configuration framework
At the end of GroupVPN initialization: Each node of a private virtual cluster gets a DHCP address on virtual tap interface A barebones cluster Additional configuration required depending on middleware Which node is the Condor negotiator? Hadoop front-end? Which nodes are in the MPI ring? Key frameworks used: IP multicast discovery over GroupVPN Front-end queries for all IPs listening in GroupVPN Distributed hash table Advertise (put key,value), discover (get key)
42 Configuring and deploying groups
Generate virtual floppies Through GroupVPN Web interface Deploy appliances image(s) FutureGrid (Nimbus/Eucalyptus), EC2 GUI or command line tools Use APIs to copy virtual floppy to image Submit jobs; terminate VMs when done
43 Classes in FutureGrid
Classes are setup and managed using the FutureGrid portal Project proposal: can be a class, workshop, short course, tutorial Needs to be approved by FutureGrid project to become active Users can be added to a project Users create accounts using the portal Project leaders can authorize them to gain access to resources Students can then interactively use FG resources (e.g. to start VMs)
44 Use of FutureGrid in classes
Cloud computing/distributed systems classes U.of Florida, U. Central Florida, U. of Puerto Rico, Univ. of Piemonte Orientale (Italy), Univ. of Mostar (Croatia) Distributed scientific computing Louisiana State University Tutorials, workshops: Big Data for Science summer school A cloudy view on computing SC’11 tutorial – Clouds for science Science Cloud Summer School (2012)
45 Deployed Systems
PlanetLab bootstrap overlays Grid appliance deployments: Archer - ~700-CPU cluster SocialVPN deployments: Thousands of downloads, hundreds of deployed nodes
On-going Work
Integration of IPOP with IPsec for dynamically- provisioned cloud virtual networks Collaboration with Thilo Kielmann (VU), Guillaume Pierre (Rennes) and the Contrail/ConPaaS teams Overlay by-pass, integration with OpenFlow software-defined networks IPv6/IPv4 overlays, virtual clusters for high- throughput computing, education Archer (computer architecture) FutureGrid (virtual appliances for education) PRAGMA (Pacific Rim Grid)
47 Acks and Thanks
ACIS P2P group (IPOP) Over the years: P. O. Boykin, Heungsik Eom, Arijit Ganguly, Pierre St. Juste, Kyungyong Lee, Yonggang Liu, Girish Venkatasubramanian, David Wolinsky, Jiangyan Xu FutureGrid, National Science Foundation Awards 0751112, 0758596, 0910812 Vrije Universiteit and Contrail For more information and downloads: http://socialvpn.org http://futuregrid.org http://grid-appliance.org
48
49