INF570
Introduction
dario.rossi
INF570 Dario Rossi v08/2013 http://www.enst.fr/~drossi Agenda
• Internet • At a glance • Internet history • Internet architecture • Applications • Terminology • Architectural models • P2P applications • A primer • Outlook on the course • References INF570
Internet dario.rossi Internet
• Internet definition ? – Internet en 2001, http://www.caida.org/ Internet
http://www.livinginternet.com/i/ia_myths_toast.htm Internet = components • Billions hots – PCs servers, phones, toasters – Equiped wtih communication capabilities • Communication links – Optical fiber, copper, air ,… carying bits and frames • IP routers – Interconnecting host, and routing data • TCP segments – Controlling data exchange and reception for all applications • Application messages – Exchanged among hosts to implement end-user services Internet
http://www.livinginternet.com/i/ia_myths_toast.htm Internet = applications + users • Application list always growing – Remote access, file-transfer, e- mail, instant messaging, Web, video and audio streaming, gaming , cloud, social networking IP/TCP • User-base always growing – > 2 billion users (12/2011) ht tp://www.internetworldstats.com/stats.htm Internet
Internet = standards • Internet Engineering Task Force (IETF) http://www.ietf.org • IETF Request For Comments (RFC ) – Technical reference documents on the TCP/IP world (HTTP, TCP, IP…) – Initially informal documents, today de facto standards – > 6000 RFCs (09/2012) http://www.ietf.org/download/rfc- index.txt http://javvin.com/map.html • Other application-specific fora – W3C pour WWW, BEP pour BitTorrent … • No documentation: – Proprietary protocols: KaZaa, Skype, etc. Internet
Internet = architecture V. Cerf and R. Kahn, “A protocol for • Offering reliable application- packet network intercommunication“ independent end-to-end services IEEE Transactions on Communications , May 1974 • Over point-to-point unrealiable individual channels • Principles : – Minimalism, best-effort, stateless core, decentralized edge control
• End-to-end – IP Routing, TCP transport, user application • Point-to-point – IP Forwading, MAC & PHY communication Internet
Internet = complex ecosystem ISPs (telcos) content providers services (free/paying) users
This viewpoint out of the scope of this course (but see readings) Internet History
• 1960s : L. Kleinrock, (Internet size) P. Baran packet-switching • 1969: 1 node
• 1970s : ALOHAnet , Ethernet • 1972: 15 nodes ARPAnet • 1983 : TCP/IP • 1979: 200 nodes • 1988 : Congestion control • 1980: 100K nodes • 1990s : Web • 1990: 1M nodes • 2000s : P2P • 2000: 1B nodes • 2010s : cloud, video (YouTube 2 nd search engine), social net (Facebook 4th country) Internet architecture Internet layers
Data unit Layer Nb – Example protocol
– HTTP, DNS, RIP, Message Application L7 BitTorrent, Skype,etc.
Segment Transport L4 – TCP, UDP, SCTP, RTP, etc. Datagram Internet L3 – IP, ICMP, IGMP, OSPF, BGP, etc.
Frame HostHost--toto--networknetworkL2 – PPP, HDLC , Ethernet, WiFi, etc. Internet operation mode
Network taxonomy Internet
Circuit switching Packet switching
FDM TDM Virtual circuit Datagram
Network IP Transport TCP UDP Circuit switching – Signaling: resource (frequences, temporal slots, etc.) allocated during connection set-up and released at tear-down by all intermediate devices – Switching : based on circuit ID, known to all intermediate equipment, fixed during all transmission – Transmission : data follow same path (i.e., the circuit), no delay – Example : Global System for Mobile (GSM), Public Switched Telephone Network (PSTN), ISDN, SONET/SDH application 6. Receive data application transport 5. Data flow begins transport network 4. Call connected 3. Accept call network data link 1. Initiate call 2. incoming call data link physical physical Packet switching
– Datagram mode • Signaling: unnecessary, each packet carries control information • Switching : istantaneous for each packet; intermediate devices unaware of ongoing transmissions (scalability) • Transmission : no performance guarantee – Virtual circuit mode • Per-flow end-to-end signaling to emulate circuit-switching (TCP); just guarantee in-order delivery, no performance guarantees
application application transport transport network network 1. Send data 2. Receive data data link data link physical physical Circuit vs packet switching
• Circuit switching • Packet switching – Resources allocated to users – No allocation, ever – Signaling necessary for connection – Flexible use of all resources on set-up and teard down a first come, first serve basis – In network-state for all ongoing – Packets may be buffered to conversations wait for their service
– No performance guarantees – Guaranteed performance (not even for virtual circuit) – No buffering – Variable delay – Easy planning for simple services – More efficient for varying traffic – Potential resource waste demand (statistical multiplexing) – Scalability problems – Simple, scalable core Internet delay Delay source at each IP hop: – Processing (< µs) - Transmission = P/C ( µs – ms) – Buffering ( µs – s) - Propagation = D/V (1 ms – 0.5s)
Variable components Fixed components (for different packets of the same flow) (for all packets of the same flow)
Processing Transmission P=packet size C=link capacity D=distance V=transmission speed
Buffering Propagation (“Bufferbloat” if very large; see later on) Internet in this course
application application application transport transport transport network network network data link data link data link physical physical physical
Application : taxonomy today (P2P apps later on) Transport: quick recap on TCP (similar problems later on) Network: quick recap on IP routing (routing at overlay later on) Internet in this course
applicationapplication application transport transport transport application network network network transport data link data link data link network physical physical physical data link physical
Application: data producer and consumer Transport: when and how much data to send Network: where to send data (short + long timescales) Transport: TCP & UDP
• UDP TCP TCP – Datagram, unrealiable, connectionless service SYN – Bitrate and message-size Round flexibility (≤ MTU) Trip -way Time SYN+ACK
• TCP (RTT) Three Three handshake handshake – Virtual circuit, connection-oriented ACK + GET with inorder and reliable delivery – Imposed tx speed (next slide) and RTT segment size (preferably =MTU) Data + FIN data User
– Initial delay due to connection setup FINACK
time Connection tear-down Transport: TCP & UDP
TCP • TCP data transfer TCPTCP – A t any tme, the tx speed is w = min(cwnd,rwnd) RTT – Rwnd = receiver window, flow control (avoid tx more than rx can handle) RTT – Cwnd = congestion window, congestion control (avoid tx more than the network RTT can sustain) transfer transfer Data Data • Cwnd dynamic Time – Algorithms: slow-start, congestion avoidance, fast-recovery, fast-retransmit, timeout,etc. cwnd Congestion Packet – Flavors: NewReno, Cubic, Compound avoidance losses – Slow-start (cwnd
• At each data packet – Decide the next hop, based on the destination address • Short timescale: istantaneous switching function – At low level implemented as a Forwarding Information Base (FIB) lookup • Longest prefix matching • Ternary Content Addressable Memory (TCAM) Network: IP routing
• Periodically – Exchange topological information between IP routers, to build FIB used in fowarding – Taxonomy: • Link state: local information to all nodes (OSPF) • Distance vector: global information to neighbors (RIP, BGP) – Internet: inter-AS + intra-AS routing
Link state Distance vector Internet structure
• Three-tiered architecure – Tier-3 (local ISPs) Tier-3 – Tier-2 (regional/national Tier-2 ISPs) – Tier -1 ( international ISPs ) Tier -1 • Full mesh between Tier-1 • Interco business model – Network access point – Customer-provider – Peering Internet structure On their end-to-end trip, IP packets traverse (i) multiple AS, and (ii) multiple routers per AS
Virtual Tier 3 local ISP local local ISP ISP ISP ISP Tier-2 ISP Tier-2 ISP Tier 1 ISP
Tier 1 ISP Tier 1 ISP Tier-2 ISP local Tier-2 ISP Tier-2 ISP ISP local local local ISP ISP ISP Internet references
• Organisations – http://www.isoc.org • Internet Society – http://www.ietf.org • IETF (Internet Engineering Task Force) – http://www.w3c.org • World Wide Web Consortium – http://www.ieee.org • IEEE (Institute of Electrical and Electronics Engineers) – http://www.acm.org • ACM (Association for Computing Machinery) • Optional reading – « A brief history of the Internet », by those who made the history http://www.isoc.org/internet/history/brief.shtml « The Internet Ecosystem », http://www.isoc.org/pubpolpillar/docs/internetmodel.pdf INF570
Internet dario.rossi Applications Agenda
• Terminology • Applications & Protocols • Application-centric vs network-centric architectures • Application: Client-server, Content distribution networks, Peer-to-peer • Network: IP Multicast, Content centric networking • Application vs transport layer • Interactions • Interface (socket) • Applications performance • Data vs multimedia (voice, video) • Quality of Service (QoS) vs Quality of Experience (QoE) • Implication on lower-layer choices • References Internet applications
User • Ultimate reason why Internet exists application • Several types: transport Router network – Remote access, email, file data link transfer, newsgroups, Web, physical multimedia (voip, videoconf), AS1 video on demand ( vod ), video streaming, social network, etc. AS2 • Internet “Killer” application Billing – Text-based (1980s), Web (1990s) Server Video P2P (2000s), Cloud/Social Server application (2010s): transport application network • Internet applications transport data link – Distributed over multiple systems network physical data link – Application communication is key physical Applications terminology
• “Application” is a misused Architecture or at least imprecise term Host – Is the email application your User agent favorite email software ? Socket
Protocol • We will be defining – Procol Socket – Architecture Daemon – Process Server • agents et daemons – Inter-process communication • sockets Networked applications • Two processes running on two • Ultimately, applications are physically separate hosts running processes of some operating system – Exchange applications messages – Of an application layer protocol – Over sockets (i.e., OS level API of TCP/IP Internet) • Two processes/threads can communicate on the same Host Host host via Inter-process Alice Bob communication Socket Host • Special case X IPC – Host-only networking – Two processes, one host • This is not the case we’re – Sockets running over the logical interested in loopback interface INF… Host INF570 X Socket Protocols
• Define – Message type application • Request, response transport – Message syntaxe network • Fields format, encoding, data link endianness,.. physical – Message semantic • How to interpret a received msg, … – Communication rules • How to respond to received (or inferred lost) messages
• Protocol definitions – public domain application • IETF RFC (ex HTTP, SMTP for client- transport server, ALTO, PPSTREAM for P2P) application network • Other fora (e.g., Gnutella, BitTorrent transport data link BEP, etc.) network physical – proprietary data link • ex Joost, KaZaA, Skype, physical Architecture • Terminology
Logical L4/L7 Service consumer communication Service producer
Infrastructure Internet physique
Local/regional nets
Servers
33 Architectures: Client-server • Client-server paradigm – Scalability problem
34 Architectures: Client-server
• Notes client • The client-side of an host communicates with the server side of another host application transport • Client-server applications can also network be highly distributed (SMTP, DNS) data link • A process may/may not implement physical all the protocol specification (i.e., client and server parts) • Examples • Web : – Web browser = HTTP client = user agent application – Web server = HTTP server = daemon transport – Web proxy = HTTP client and server network • Email : data link physical – Mail composer = SMTP client = user agent Internet application generally rely on two entities: clients and servers – Tx/Rx server= SMTP client/server = daemon Architectures: CDN • Content distribution network (CDN) – Costly infrastructure, application-specific service
36 Architectures: CDN
Client • Content distribution networks – Business plan: content application transport replication as close as possible to network the users data link – Good for ISPs: avoid costly inter- physical AS transit links – Good for users: close content, low delay, high intra-AS link bandwidth CDN • Examples – Videos YouTube (pre-google era), application Netflix (uses 3 CDNs), patches application transport Microsoft & Antivirus, … transport network network data link 28 CDNs operators worldwide, – data link physical biggest share Akamai, Limelight, physical Architectures: Multicast, CCN • IP Multicast, Content Centric Networks (CCN) – Complex (multicast used for IPTV; CCN = research)
38 Architectures: Peer-to-peer
• P2P Paradigm R Application layer (easy deployment) R Use all resources (scalable) R Use edge resources (shift costs)
39 Architectures: Peer-to-peer
Peer peer • Playing both the client and server roles • Client: ask service to other peers application transport • Server: offer services to other peers network data link Notes physical • Servers are managed by service provider • Protocol can assume servers to be always available (except troubles) • Peers are user workstation • Protocols must assume peers not always available application • Most tasks of P2P applications are transport heavily distributed network data link • Some exceptions (e.g., bootstrap) physical Architectures: summary
Application-centric Network-centric • Client-server • IP Multicast • Content distribution • Content centric networks network (CDN) (CCN) • Peer -to -peer (P2P)
• Over-the-top solution • Potentially more efficient (“patches” to cure the • Very slow deployment Internet) (see the rolling of IPv6) • All software solution (download & run) L7 overlayArchitecture: vs IP underlay Peer-to Interaction-peer • Internet applications (L7 OSI) build on top of the existing TCP/IP underlay infrastructure
P2P overlay
IP underlay
42 L7 overlay vs IP underlay Interaction
• Internet applications (L7 OSI) build on top of the existing TCP/IP underlay infrastructure
L7 overlay Interaction L7/L3: applications vs TCP/IP layers IP underlay
43 Interaction L7 overlay vs IP underlay
• Traffic control (e.g., “network friendly” if competition is fair with respect to standard TCP congestion control)
L7
overlay L4-like problem. Yet, solutions differ from TCP (e.g., Skype, BitTorrent, etc.) IP underlay
44 Interaction L7 overlay vs IP underlay
• Routing (“network aware” if traffic is confined as much as possible within AS boundaries)
$$ L7 overlay L3-like problem. Yet solutions differ from IP, RIP, OSPF et BGP IP underlay AS 1 AS 2
45 Socket • Application program interface (API) – offered by the Operating System (OS) for TCP/IP internetworking – Entry/exit point for all networking data – Interface between application layer protocol and TCP/IP • Developer has little-to-no control over the sockets – L4 protocol choice + few parameter tuning, especially in user mode
Controlled by application Software process process developer/user socket socket Controlled by operating TCP/IP TCP/IP system and super-user Software (root) driver Internet driver Controlled by NIC NIC manufacturer/OS Hardware NIC = Network interface card Socket
• Addresses • Human-to-network translation – Human viewpoint URL – Domain name system (DNS): • http://www.google.com:80 name to IP address – Network viewpoint – Rules for port number • L3: IP address (/etc/services ) • L4: protocol type – 5-tuple ( IPs,Ps,IPd,Pd,proto ) • L4: port number unambigously identify a stream of data between two hosts (TCP flow, UDP segment sequence)
[email protected] http://www.google.com
TCP/IP Internet TCP/IP
137.192.164.1:37562 Internet 74.125.39.180:80 Transport-layer addresses • Port number: 16bits integer (=65K), two functions Assume a client C has 2 parallel HTTP downloads • Multiplexing (ephemeral port numbers > 32K) from the same server S: – Disambiguate processes (TCP:C: p1 :S:80) (TCP:C: p2 :S:80) • Resolution (reserved numbers < 32K) – Simple deterministic addressing at L4 (no need Once DNS resolves the name to IP address , the for a second resolution beyond DNS) port number is already – Well-known port numbers : RFC 1700, /etc/services known without waiting • E.g., 80=Web, 53=DNS, etc. for a second resolution • Note – New numbers assigned by IANA (apps typically comply) – Web severs may run on non-standard ports (eg. 8080), P2P traffic may run on arbitrary ports (eg. 80, 53) – P2P applications no longer use fixed hardcoded ports Socket: which L4 protocol?
• TCP & UDP offer different • Many transport layer transport-level service protocol exists
– UDP: User Datagram Protocol application – RSVP: Resource reSerVation transport Protocol network – SCTP: Stream Control data link Transport Protocol physical – DCCP: Datagram Congestion Control Protocol As well as many flavors – T/TCP: Transaction TCP for the same protocol – RUDP: Reliable UDP • TCP NewReno (IETF) – TCP: Transport Control Protocol • TCP Cubic (Linux) – etc. • TCP Compound (Windows) • LP, Vegas, NICE, Westwood, ... Socket: which L4 protocol? • Flows content differ Packet size (Bytes) – Data • Wb • e-mail HTTP flow Size determined by • filesharing TCP: packet trains having • ... MSS-size, followed by – Multimedia ACK -size packets • Video (on demand, broadcast) • Telephony • ... Skype “flow” Size determined by the • Flows requirements voice encoder (variable therefore differ bitrate) – Choice determined by application performance Packet no. Socket: which L4 protocol? • Flows content differ Inter-packet gap (sec) – Data • Wb Time determined by RTT • e-mail HTTP flow • filesharing (~20ms) between 2 trains, • ... Packets are back-to-back (~0ms) within a train – Multimedia • Video (on demand, broadcast) • Telephony • ... Time determined by the • Flows requirements voice encoder (variable therefore differ Skype “flow” bitrate) – Choice determined by application performance Packet no. Socket: which L4 protocol? • Flows content differ • L4 Connectivity – Data – Another important piece of the puzzle • Wb • TCP • e-mail • filesharing – Usually associated with good, • ... legitimate, responsive traffic – Hence, port 80 (HTTP) and 443 (HTTPS) – Multimedia rarely blocked • Video (on demand, • UDP broadcast) – Unrsponsive (illogically) associated with • Telephony • ... dangerous activity – Hence, only port 53 (DNS) open • Flows requirements • Generally therefore differ – Application have a preferred transport layer protocol (but can failback over TCP – Choice determined by application performance in case of connectivity problems) Application performance
Quality of Service (QoS) Quality of Experience (QoE) (network-centric) (user-centric) • Throughput • Data oriented Bits per unit time – Completion time (time to last byte) • Loss rate – Reactivity (time to first byte; Probability that a packet arrives interactive applications ) with detectable but not – Reliability correctable errors – … (wireless) • Multimedia Probability that a packet is lost – Peak Signal to Noise Ratio (PSNR) at a full buffer (wired ) – Mean Opinion Score (MOS) • Delay – Structural similiarity (SSIM) Packet travel time – Percetpual Evaluation of Video • Jitter Quality (PEVQ) Delay variability – …
Easy to measure, objective, not Complex, costly, application very telling for app. performance specific, no worldwide agreement Application performance • Client-server • Peer-to-peer C:c A:a B:b… N:n S:sTCP
1 1
2 3 3 4 4 2 5 5 • Single-flow service • Multi-flows service – Incoming traffic (L4:S:s:C:c) 5-tuple – Incoming traffic (L4:*:*:A:a) aggregate – Only consider temporal dimension – Importance of spatial dimension – Oversimplification; see HTTP parallel connections, or DASH streaming) – Application performance depend on the flow aggregate Data traffic
• Examples – Client-server: Web, data exchange, remote access, e-mail, … – Peer-to-peer: filesharing, cooperative file system, lookup … • Primary metric: reliability – No tolerable error (e.g., binary data), in-order arrival • Secondary metric: throughput – Tolerant, can exploit available capacity (elastic application) • When requiring interaction: delay – Web: tolerable delay max 3 seconds (else “World Wide Wait”) – Remote terminal, googledocs: 20ms delay can be perceived – Filesharing: order of seconds (if requiring any user interaction) – File system: listing requires sub-second delay, transfer is elastic Streaming traffic • Examples – Client-server: Dynamic Adaptive Packet Streaming over HTTP (DASH), sequence Source YouTube number – P2P: P2P-radio (audio), PPLive, SopCast (TV), Miro (VoD) Deadline Receiver • Primary need: temporal coherence Frozen stream – Throughput >= stream rate • Secondary need: low lag Time – Especially for live radio/TV Delay – Tradeoff low lag vs freeze Playout buffer (lag w.r.t. source) Voice traffic
• Digital transmission • Other considerations: – Vocoder: Sampling, quantization and encoding – Slow, redundant signal, robust to losses – Application: Framing, transmission, forward error • Perceptual quality correction vs loss recovery tolerable with 15% losses – No longer resilient to losses • if lossy vocoder already • Primary metric: delay suppressed redundancy – Must limit end-to-end delay • Cannot use buffering – Maximum “mouth-to-ear” • Remember delay: – Even if you design a non-VoIP • can tolerate 300 ms application, you may find one • 45 ms without echo in your way suppression – see course on LEDBAT • Secondary metric: jitter – Keep jitter low Voice encoder
Pulse Code Modulation (PCM) • Vocode output – Low bitrate • Sampling • PCM : 64 kbit/s (4 kHz spectrum => Nyquist 8000 • GSM & iLBC (Skype) : 13,3 samples/sec) kbit/s – Small packets • 80 bits - 1 kbit – Inter packet gap • 20 – 100 ms • Quantization (256 levels, 8 bits/sample) • Silence suppression – pink noise/noise descriptor to avoid unnatural silence 01001100 11001111 • Differential encoding time – reduce quantization bits • Bitrate • Lossy compression 64kbps (1 byte every 125 µs ) – suppress frequences less important for human ears Video encoder
• Important bitrates MPEG – TV : 720 x 576 pixels 16bpp, 25 fps – Suppress spatio-temporal redundancy – 166 Mbps raw stream – I-frames : “Intracoded” full JPEG picture (~1I/sec) • Compression is a must – P-frames : “predictive”, delta wrt previous – MPEG1: VCD quality 1-2 Mbps – B-frames : “bidirectional”, wrt previous and – MPEG2: DVD quality 5-10 Mbps following (two -pass encoding) MPEG4: HDTV quality 20-60 Mbps – – D-frames : “DC coded”, useful for fast- – H264, SVC, MDC, ... (new codecs, forward (~1D/sec) multiple streams, not in this course)
Highly variable bitrate II II PP PP BB BB BB BB BB BB Other traffic
• Online gaming – Non-service specific flows – Delay sensitive (100 ms, end-to-end) • All applications, directly or – Low bitrate indirectly , depend on them! – Indirectly: • Multimedia flows Network maintenance – Several flows (video + audio + subtitles) • Control information, – Synchronization routing, etc. • Operation and management, • Distributed jam-session monitoring, etc. – Real time playback – Directly: (50ms maximum, <20 preferably) Application-layer – Heterogeneous bitrate: very low (MIDI) to “transparent” services high (6 Mbps sequencing @192 kHz, 32bit) Domain name translation DNS – Example applications: Musigy, eJamming, • • Signaling, security, etc. Application summary
Service Protocol Transport Control Addressing DHCP, DNS data-oriented, UDP delay before reliability Web HTTP TCP - VoD YouTube bang -bang over TCP interaction L7/L4 P2P * often both - P2P TV Pplive, SopCast UDP, TCP P2P DHT Kad UDP P2P VoIP Skype UDP, TCP if needed L7 control P2P Filesharing Emule UDP P2P Swarming BitTorrent UDP since 2010 L7 control QoS vs QoE in practice
• Apply controlled QoS settings – E.g., artificial loss, bandwidth limit, variable delay • Observe application response – Perceptual QoE (~ Mean Opinion Score, MOS)
– Objective QoS metrics Today, or in Lab-02 (Experiment)
Network emulator App App App App App App Loopback Internet Host only Network LAN networking + emulation INF570
P2P Applications dario.rossi Agenda • Five Ws and one H (*) – P2P ecosystem evolution 2000-nowadays – Relevance w.r.t. rest of Internet applications – Interest of P2P paradigm (vanilla model) – P2P networks and Overlay graphs – Taxonomy of P2P applications
(*) 5W (Who, What, When, Where, Why) and one H (how) This intro Rest of the course Who, What, Why, Where & When
• Who (=P2P) • What (=Hype and widespread growth) • Where (=Everywhere; not discussing geographical P2P popularity) • When (=Next slides, with “how much?” as a bonus)
• Why ? – Empower the users • Well, especially the malicious ones – Low deployment barrier • often free, not always open-source, click-and-run software – Introduce new business models • By shifting costs, everybody becomes content producer (interesting example in P2PTV/PPLive) P2P ecosystem evolution
2005 1999 2000 2001 2002 2003 2004 2007 2008 2011 t
Napster Gnutella Kazaa eMule Skype PPLive Sopcast Joost uTorrent 3.0
Cool Bittorrent VoIP streaming TVAnts
Limewire Live streaming
File-Sharing Megaupload Spotify Music Chord Kademlia streaming
P2P in Search Web based browsers
21/09/2011 P2P traffic evolution Oversimplifying – Absolute volume increasing since 2000 – Around 2004, P2P seemed to vanish... but was hiding! – Traffic classification become a hot research topic – Relative volume decreasing in recent years – Rest of the Internet traffic increasing faster (esp. VoD, filehosting) – Growth of mobile Internet (>50% traffic), few P2P on mobile – By the way, “volume” – Bytes ? BitTorrent is champion – Flows ? Skype, PPLive, ... – Future hard to predict at any point – Megaupload shutdown ? More P2P! – P2P in HTML ? More P2P!
2000 2005 2010 P2P traffic: some numbers
Ipoque , Internet studies 2008-2009
Sandvine Global Broadband Phenomena 2009 Cisco Visual Networking Index 2010 Arbor Networks 2009 19.30% 3.28% 5.51% 33.70% 31.27% 29.65% 23.07% 45.72% 20.40%
18.32% 26.60% 21.35% 21.83%
Web Video P2P Web Video P2P Web Video P2P Other Unclassified Others Unclassified Other Unclassified P2P traffic: our numbers
HTTP Steep increase of YouTube BitTorrent UDP… BitTorrent ISP1 … Other TCP Other UDP BitTorrent UDP …
eMule ISP5
69 Client-server paradigm
Client Server – Runs on end-hosts – Runs on end-hosts N-1 – On/off behavior – Always on N – Service consumer – Service provider 3 – Issue requests – Receive requests
– Do not communicate – Satisfy all client 2 directly among them requests 1 S – Need to know the – Need a fixed IP server IP address (or DNS name) Peer-to-peer paradigm
Peer – Runs on end-hosts N-1 N – On/off behavior, handle churn
– Need to join 3 – Need to discover other peers – Service providers and consumers 2
– Communicate directly among them 1 S – Need to define communication rules • Prevent free riding • Incentivate participation ... and reciprocation Peer-to-peer paradigm
Servers used Peer for bootstrap S X – Runs on end-hosts
– On/off behavior, handle churn X A B – Need to join S C ? – Need to discover other peers D – Service providers and consumers – Communicate directly among them E – Need to define communication rules ? • Prevent free riding B • Incentivate participation A and reciprocation D , E Client-server vs Peer-2-peer • Interest of P2P
N-1 N-1 N N
3 3
2 2
1 S 1 S
What is the minimum download time for the latest peer under either paradigm ? (vanilla model) Client-server
N-1 F bits long file N 1 server Us server upload rate N clients 3 Di download rate of the i-th client Dmin = min( Di ) , slowest client Ti download time of the i-th client 2 T = max( Ti ), system completion time Assuming simple fluid model, server policy 1 S is to give each of the N peers an equal share of its rate Us/N, what is T equal to ?
(1) T ≥ F / (Us/N) = NF / Us i.e., the download cannot be faster than the share of the server upload capacity allows (2) T ≥ F / Dmin i.e., the download cannot be faster than the downlink capacity of the slowest peer allows T ≥ max( NF/Us, F/Dmin ) = F/min( Us/N, Dmin ) Peer-2-peer
N-1 F bits long file N 1 source peer (has the content at time t=0) Us source peer upload rate N sink peers 3 Ui,Di upload, download rate of the i-th peer Dmin = min( Di ) , slowest peer Ti download time of the i-th client 2 T = max( Ti ), system completion time Assuming simple fluid model, source gives bits 1 S to peers at rate Si, each peer replicate received data toward the others N-1 peers at rate ≤ Si (1) T ≥ F / Us i.e., no peer can receive faster than the source can send (2) T ≥ F / Dmin i.e., the download cannot be faster than the downlink capacity of the slowest peer allows (3) T ≥ NF /( Us + ΣUi ) i.e., the overall data cannot be downloaded faster than the aggregated system capacity (with peers) allows T ≥ F/min(Us, (Us + ΣUi)/N, Dmin) Client-server vs Peer-2-peer Interest of P2P •P2P protocols can offload server capacity, allowing better scaling •Example when resource=capacity and service=diffusion, conclusion hold for many resources/services •Note: vanilla model, other flavors later
800 peer -2-peer (sec) (sec) 700 client -server 600 500 File size F=10MB, Peer upload Ui=500 Kbps, 400 Source upload Us=10 Mbps 300 Peer download Di >> Ui 200 100 File diffusion time time diffusion File Tmin 0 10 20 30 40 50 60 70 80 90 100 Number of peers/clients N P2P Overlays
• P2P network is a graph, where edges represent logical connections between peers
• Logical connection – Ongoing communication via TCP , UDP sockets – Reachability within application layer routing tables P2P Overlays • P2P networks are commonly called Overlays as they are laid over the IP infrastructure
AA B • Not all logical links Application are also physical links C Overlay • Not all physical links are necessarily used • IP is not a physical network but a logical mesh (IP is an L3 overlay) • Examples of L7 overlay IPa IPb Physical • client-server: SMTP, DNS Network • CDN: Akamai, limelight IPc • P2P: later in this course P2P Overlays • P2P overlay graphs are very much different, and depend on implementation choices
App1 App2 P2P Taxonomy
• P2P services – File-sharing and content distribution Live TV / VoD – Voice/video call Filesharing – Television/VoD – CPU sharing BitTorrent eDonkey SopCast TVAnts PPLive Joost – Indexing and search • P2P taxonomy VoIP/Chat Search SW Updates – Structured RapidUpdate DebTorrent – Unstructured Skype Gtalk Kademlia Apt-p2p,… – Hierarchical – Structure is not tied to a particular service Unstructured Overlay • Peers – arbitrarily connected • Lookup – typically implemented with flooding (or out-of-band) • Pro/Cons – Easy maintenance, diffusion – Highly resilient – High lookup cost • Examples – Gnutella (up to v0.4) – BitTorrent, P2P-TV systems (out-of-band lookup) Hierarchical Overlay • Peers and super-peers – Ps connect to super-Ps – Super-Ps know Ps resources • Lookup – Flooding restricted to super Ps • Pros/cons – Lower lookup cost, – Improved scalability – Less resilient to super-Ps churn • Examples – Skype (for relay), eDonkey (pre Kad), Gnutella (v0.6 on) Structured Overlay • Peers – Carefully connected shaped as a precise overlay topology • Lookup – Overlay routing algorithm are not based on flooding – Takes advantage of regularity • Pros/cons – Effective lookup & great scalability – Complex maintenance • Examples – Distributed Hash Tables (DHTs) P2P challenges • Peers – Come and go due to their will or due to failures – Have/have not the resources you are looking for – May/may not be willing to give the resource
• Challenges – Effectively locate the resources • Routing, forwarding – Effectively share the resource users • Incentivate peers participation to the system, scheduling – Scale to several million users, face churn, work in practice • Overlay management, peer selection, network awareness Focus of this course
– Lookup and routing • Find the peers having the resource of interest (e.g., a file or a contact for messaging/calls) – Diffusion and scheduling • Ensure timely and effectively diffusion of a resource (e.g., a file, or a TV stream). – Transport and congestion control • Interactive, near real-time applications (e.g., Skype) • Background applications (e.g., BitTorrent) ?? || //