Informed Content Delivery Across Adaptive Overlay Networks John Byers Jeffrey Considine Michael Mitzenmacher Stanislav Rost
[email protected] [email protected] [email protected] [email protected] Dept. of Computer Science EECS MIT Laboratory for Boston University Harvard University Computer Science Boston, Massachusetts Cambridge, Massachusetts Cambridge, Massachusetts Abstract Categories and Subject Descriptors C.2 [Computer Systems Organization]: Computer- Overlay networks have emerged as a powerful and highly flexible Communication Networks method for delivering content. We study how to optimize through- put of large transfers across richly connected, adaptive overlay net- General Terms works, focusing on the potential of collaborative transfers between Algorithms, Measurement, Performance peers to supplement ongoing downloads. First, we make the case for an erasure-resilient encoding of the content. Using the digital Keywords fountain encoding approach, end-hosts can efficiently reconstruct Overlay, peer-to-peer, content delivery, digital fountain, erasure ¡ the original content of size ¡ from a subset of any symbols drawn correcting code, min-wise summary, Bloom filter, reconciliation, from a large universe of encoded symbols. Such an approach af- collaboration. fords reliability and a substantial degree of application-level flex- ibility, as it seamlessly accommodates connection migration and 1 Introduction parallel transfers while providing resilience to packet loss. How- ever, since the sets of encoded symbols acquired by peers during Consider the problem of distributing a large new file across a downloads may overlap substantially, care must be taken to enable content delivery network of several thousand geographically dis- them to collaborate effectively. Our main contribution is a collec- tributed machines.