Πολυμεσικό Υλικό στο Internet: Συγχρονισμός, Επεξεργασία και Διακίνηση Multimedia content delivery in the Internet: CDNs, Real Time Streaming, Multimedia platforms & video service providers Β. Μάγκλαρης <[email protected]> Μ. Γραμματικού <[email protected]> Δ. Καλογεράς <[email protected]> www.netmode.ntua.gr Outline • Content Delivery Networks (CDNs) • Video on Demand servers/clouds • Real Time Streaming Content Delivery Networks (CDNs) http://en.wikipedia.org/wiki/Content_delivery_network • The first generation CDNs were not encountered before the late 90′s • CDN is a large distributed system of servers deployed in multiple data centers across the Internet • It serves content to end-users with high availability and high performance • It serves a large fraction of the Internet content : – including web objects (text, graphics and scripts), downloadable objects (media files, software, documents), applications (e-commerce, portals), live steaming media, on-demand streaming media, and social networks Single server distribution and CDN distribution CDN Providers uses different Business Model • Google maintains it’s own delivery network consisting of large number of GGC (Google Global Cache) nodes placed on ISPs network and help in serving Google’s static content • Akamai (whose core business is into Cache delivery) put their servers on large number of edge networks but they stay as disconnected small islands • New comers in the industry like Limelight, Cloudflare put node in major datacenter and direct connection to major networks via peering from IXPs Notable Content Delivery Service Providers http://en.wikipedia.org/wiki/Content_delivery_network Free CDNs[edit] Telco CDNs[edit] •BootstrapCDN •AT&T Inc. •CloudFlare •Bharti Airtel •Coral Content Distribution Network •Bell Canada •Incapsula (free version with Incapsula advertisement) •BT Group Traditional commercial CDNs[edit] •Deutsche Telekom •Akamai Technologies •Hibernia Networks •Amazon CloudFront •KT (formerly Korea Telecom) •Aryaka •KPN •Windows Azure CDN •Level 3 Communications •CacheFly •MegaFon •CDNetworks •NTT •ChinaCache •Pacnet •Cotendo (acquired by Akamai) •PCCW •Distil Networks •Qualitynet •EdgeCast Networks •SingTel •Highwinds Network Group •SK Broadband •HP Cloud Services •Tata Communications •Incapsula •TeliaSonera •Internap •Telecom Argentina •LeaseWeb •Telecom Italia •Level 3 Communications •Telecom New Zealand •Limelight Networks •Telefonica •MaxCDN •Telenor •MegaFon •Telstra •MetaCDN •Telus •Mirror Image Internet •Turk Telekom •NACEVI •Verizon •OnApp Commercial CDNs using P2P for delivery[edit] •OVH •BitTorrent, Inc. •Rackspace Cloud Files •Internap •Speedera Networks (acquired by Akamai) •Pando Networks •StreamZilla CDN Europe •Rawflow CDN 2014 – 2017: Operations and Analytics http://www.reportsnreports.com/reports/287274-cdn-2014-2017-operations-and- analytics.html • CDN (Content Delivery Network) revenue is projected at $3.36 billion in 2014, up 19%,directed by user appetite for self-selected content • Total 2013 commercial value of media and entertainment video (views and advertising), movie/TV files, music listening and downloads (including self-hosting entities such as Google and Amazon) stood at $3.35 billion, of which $1.05 billion (31.3%) was delivered through CDN contracts • Video viewing and advertising (combining self-hosted networks), TV/ movies and music accounted for 2.4+ billion gigabytes of data transfer, worth $1.6 billion in commercial market value (bandwidth and co-lo fees only), when priced at prevailing rates and against a volume (gigabytes delivered) model • CDNs include Akamai Technologies, Limelight Networks, Cd Networks, China Cache, Mirror Image, Level 3, High winds, Cloud flare, Tata Communications and MaxCDN, among others Published By: AccuStream Research Content Delivery Networks http://www.globaldots.com/the-history-of-content-delivery-networks-cdn • In the past, the development of CDNs sought to deal with extreme bandwidth pressures, first as video streaming was growing in demand as well as the number of content providers • Now, CDNs are a continual trend, with the emergence of utility - cloud computing, charging end-users on demand and involving all layers of cloud computing: – SaaS (Software as a Service), e.g. Google Docs, Amazon CloudFront – PaaS (Platform as a Service) e.g. Google App Engine – IaaS (Infrastructure as a Service), e.g. GRNET Okeanos, Amazon EC2 Layers of Cloud Computing http://www.globaldots.com/the-history-of-content-delivery-networks-cdn Content Νetworking Τechniques • The Internet was designed according to the end-to-end neutrality principle. Core network : simple and moves the intelligence to the network end-points (hosts, clients) • CDNs distribute a variety of applications employing techniques to optimize content delivery. The resulting tightly integrated overlay uses web caching, server-load balancing, request routing, and content services – Web caches store popular content on servers that have the greatest demand for the content requested – Server-load balancing uses one or more techniques including service- based (global load balancing) or hardware-based • A variety of algorithms are used to route appropriately the request. These include: – Global Server Load Balancing – DNS-based request routing – Dynamic metafile generation – HTML rewriting and – Anycasting Algorithms used in Request Routing in CDNs http://www.codeproject.com/Articles/391966/Overview-of-CDN-Content-Delivery- Network • Global Server Load Balancing (GLSB) enables the content to be obtained from a server pool in a sequential manner using round- robin method and redirect the request in case of inactive server sessions • DNS-based request routing: when a request is made (URL), the local DNS server provides the IP address of the nearest matching CDN node. If the Local DNS is not able to resolve the URL, it forwards the request to the Root DNS server, which then provides the nearest possible CDN server IP • Dynamic metafile generation includes creation of a metafile, which has an ordered hierarchy of CDN domains connected to a Main server and helps in the load balancing on each of CDN nodes connected to it Protocols used in Request Routing in CDNs http://www.codeproject.com/Articles/391966/Overview-of-CDN-Content-Delivery- Network • ICAP (Internet Content Adaptation Protocol) is a high level protocol that helps in generating http requests and delivers contents from the CDN servers • OPES (Open Pluggable Edge Services) uses a Processor in order to share contents to the end users. This processor duplicates the content at each CDN node and traces the route followed by each request made by the user and notifies the user once the content is found • ESI (Edge Side Includes) avoids back end processing delays hence providing dynamic contents with ease. It breaks web content into fragments and delivers dynamic contents to end users Content Delivery Networks (CDNs) Content Delivery Networks State of the Art, Insights, and Imperatives_2008, Mukaddim Pathan, Rajkumar Buyya and Athena Vakali • Most CDNs are operated as an Application Service Provider (ASP) on the Internet (also known as on- demand software or SaaS) targeting Content Providers, web administrators etc., wishing to optimize delivery to end-users • CDN Service and Functionalities: – Storage and Management of content – Distribution of content among edge servers – Cache management – Delivery of encoded media – Backup and disaster recovery solutions – Monitoring and Performance P2P and Private CDNs • Peer-to-peer CDNs – In P2P content-delivery networks, clients provide resources as well as use them, so the content serving capacity of P2P networks can actually increase as more users begin to access the content • Private CDNs – The owners create their own CDN – A private CDN consists of Points of Present (PoPs) that are only serving content for their owner Top International CDNs make deals directly with ISPs and/or Internet Exchanges http://blog.streamingmedia.com/2014/07/apples-cdn-now-live.html https://peering.google.com/about/ • Apple’s CDN Now Live: Has Paid Deals With ISPs, Massive Capacity In Place • Google Peering and Content Delivery – Data centers – Backbone – Edge Points of Presence (POPs) – Google's edge caching infrastructure • Netflix Announces New Content Delivery Network, Offering Free Caches To ISPs Free CDN Services for WordPress Blog • CloudFare : the data centers are located all across the world to serve you the content at blazing fast speed • Photon CDN: is a content delivery network that loads images on your blog through the powerful WordPress.com servers • Swarm CDN: it offers a free plan with 250 GB of free transfer • jsDelivr: CDN service for hosting JavaScript files • Coral CDN: is a P2P (peer-to-peer) based content delivery network and a project from MIT. Billions of computers are connected across the world, making it absolutely free A Pioneer Content Delivery Network (CDN): Akamai http://www.akamai.com/html/about/company_history.html Erik Nygren, Ramesh K. Sitaraman, Jennifer Sun, The Akamai Network: A Platform for High-Performance Internet Applications • Akamai's beginnings lie in a challenge posed by World Wide Web inventor Tim Berners-Lee at MIT in early 1995 • The company launched commercial service in April 1999 • The platform is made up of a distributed network of servers and intelligent software, delivering over two trillion interactions daily • The Akamai Intelligent Platform is constantly monitoring Web conditions to: – Identify, absorb,