How to successfully peer at 20+ Internet exchanges Grzegorz Janoszka Network Design Engineer LeaseWeb B.V. Contents . Introduction . History of peering connections of LeaseWeb – from 0 to 30 . Why to peer? Why at so many exchanges? . Detailed analysis of AMS-IX, DE-CIX, LINX and PLIX . Tips and hints for cost effective peering . Is the sky really the limit? . Summary, Q&A LeaseWeb, established in 1997, is now one of the biggest hosting companies with multiple datacenters in EU and US. 70.000.000 € revenue . 40.000 servers . >10.000 customers . 1.250.000.000.000 bps traffic . 200 employees OCOM group: LeaseWeb – hosting EvoSwitch – running datacenters FiberRing – ISP, data transmission DataXenter – building modular datacenters Brief history of peering connections: . 1997 - Company founded, first servers in datacenters within Amsterdam area . 2000 - Own IP’s and ASn (16265) assigned . 2002 - AMS-IX (existing since 1994) matures and starts a professional NOC, Leaseweb joins with 1GE port . 2005 – first DE-CIX connection, also 1GE . 2007 – first LINX connection, 10GE . 2008 – 10GE: BNIX , Netnod; 1GE: PLIX, SwissIX, NYIIX, VIX, NIX.CZ . 2009 – 1GE: Espanix, SIX, MIX, BIX, Interlan . 2010 – 1GE: DIX, NIX, FranceIX, LU-CIX, FreeBIX, NL-IX . 2011 – Equinix Ashburn . 2011-2012 – ECIX, Equnix: Palo Alto, Dallas, Chicago, New York, Los Angeles, Telex, NOTA, Any2 Los Angeles, Lonap... Current capacity to Internet Exchanges: . AMS-IX – 140 Gbps . DE-CIX – 70 Gbps . LINX – 50 Gbps . 20 Gbps: Netnod, Espanix, NL-IX . 10 Gbps: SwissIX, BNIX, PLIX, FranceIX, BIX, MIX, Equinix Ashburn . 1 Gpbs: 9 others . 22 connected, 10 more to go Why to peer (publicly)? It is NOT another peering vs transit debate... ...but peering is just cheaper... at least for us Typical 10GE wave cost is about 3000€ MRC, plus ~2000€ IX fee gives ~0.5 €/Mbps With transits for ~1€/Mbps it may be cheaper, efficiency is the key. Other peering advantages: . Marketing value – our customers like it and it is very important for some of them . Better performance, increased redundancy and resiliency . Public peering can be just a prelude and a reason of a private peering later on . Provides a relatioship with a peer, it is easier to request assistance and troubleshoot problems together Almost a dozen Internet exchanges with only 1 Gbps connection while generating 1.25 Tbps total traffic... does it make any sense? . it is not always much cheaper... ...but it is important for our customers . you can learn the market, already get peers, estimate traffic and then you can upgrade to 10GE, which provides cheap traffic . almost no administrative burden because of route-servers actively used With peering connections utilized less than 50% you are not going to make any money. Efficiency of utilizing links at at least 70% of capacity is the key. Steer your traffic. Don’t upgrade too fast, first build your traffic. Multiple exchanges help greately improve efficiency. With AMS-IX, DE-CIX and LINX being full you don’t have to upgrade capacity to all of them. Instead, upgrade only one, shift some traffic towards it, thus making free space on the others. Repeat the cycle upgrading only one IX at a time. Important: be a good peer, please always contact your peers before shifting your traffic. How to make your peering cheaper? . Cooperate, swap services, share costs . If your traffic is not balanced, find someone to utilize the other direction . Negotiate with exchanges, especially the smaller ones may do a lot to have you there . Be selective, use route-servers . Consider ordering multiple services with one common commitment (next slide) What are the limits? . The number of exchanges is limited, about 100 now . Only 82 ASn’s peer at 10 or more IXP’s . Only 6 ASn’s peer at 30+ IXP’s . The leader is Google peering at 54 IXP’s Google 15169 54 Akamai 20940 50 Packet Clearing House 3856 45 Limelight 22822 41 Hurricane Electric 6939 40 Woodynet DNS Anycast 42 34 Yahoo! 10310 29 E4A 34695 28 Microsoft 8075 27 RETN 9002 24 BroadbandOne (VW Fiber) 19151 23 Mzima 25973 23 LeaseWeb 16265 22 https://www.euro-ix.net/tools/asn_common It is feasible to peer at many Internet exchanges. Scale is the key – the bigger your network is, the more exchanges you may peer at. Don’t focus only on the price, there other advantages. Peering is not a religion, always consider pro’s and con’s. Thank you very much, and happy peering! ? .