The Big Picture 2
Total Page:16
File Type:pdf, Size:1020Kb
QUESTnet 2004 18th APAN Meetings The AARNet story 1. Chris Hancock – The big picture 2. Don Robertson – The AARNet 3 rollout 3. Keith Burston – The regional network 4. George McLaughlin – The international network – Charging 5. Steve Maddocks - Operating the network 6. Mark Prior - Peering ©Copyright AARNet Pty Ltd 1 ©Copyright AARNet Pty Ltd 2 Setting the foundations •Traffic growth continues to escalate The big picture •Prices continue to fall •Our success is our strategic Chris Hancock alliances •Nextgen provides new opportunities •Charging moves away from usage ©Copyright AARNet Pty Ltd 3 ©Copyright AARNet Pty Ltd 4 Traffic continues to grow Prices continue to fall Terabytes $/GByte 900 $120.00 776 $113.78 800 $104.67 $100.00 700 577 $88.12 600 $80.00 500 440 $67.52 $60.00 400 $52.86 289 300 195 $40.00 200 125 $20.00 $22.31 100 0 $- 1998 1999 2000 2001 2002 2003 1998 1999 2000 2001 2002 2003 Year Year ©Copyright AARNet Pty Ltd 5 ©Copyright AARNet Pty Ltd 6 1 Gigabit capacity within Australia – challenges and solutions Key strategic alliances • Poor response to request to carriers to make dark fibre available • with Australian Government and the Australian Research and Education – Solution: use AARNet’s carrier licence to carry out civil works Network Initiative (AREN) – provided funding to catalyze key regional to lay our own fibre initiatives – Only needed to do once in any area after this some enough • with Leightons (construction company – dark fibre across Australia as players willing to sell/lease dark fibre basis for AARNet3 and regional gigabit connectivity (eg to remote • Poor response from traditional carriers for high capacity connectivity telescopes) to regional areas • with Powerlink (North Queensland) and TransGrid (NSW) – power utility – Solution: form strategic alliances with power utilities for mutual companies that provided fibre to regional areas – resulted in gigabit benefit capacity where not previously economically feasible – Now have affordable gigabit capacity in North Queensland and • with Southern Cross on SXTransPORT inland NSW • Tasmania: monopoly carrier, no affordable high capacity option • with AUSAID (Australian Aid Agency) to provide a connection from the Fiji Campus of the University of the South Pacific to AARNet and global – Solution: form strategic alliance with State govt, Basslink, gas pipeline groups to deploy alternative undersea cable capacity R&E networks ©Copyright AARNet Pty Ltd 7 ©Copyright AARNet Pty Ltd 8 AARNet’s Australian network - ‘world class’ AARNet a ‘world player’ ©Copyright AARNet Pty Ltd 9 ©Copyright AARNet Pty Ltd 10 Charging Catalysts for change •Moving away from a volume based •Moved from a reseller of charging model to a charging regime that capacity to owning and is largely subscription based managing significant •To encourage innovative and unconstrained use of the network for infrastructure research and education •Now managing and operating •More on charging later … the AREN ©Copyright AARNet Pty Ltd 11 ©Copyright AARNet Pty Ltd 12 2 Where to from here? • Building Australia’s next generation of advanced networks • Developing a future network road map for network The AARNet 3 rollout connectivity • Developing new services for our members Don Robertson • Reaching more of the research and education sector • Partnering to secure new initiatives ©Copyright AARNet Pty Ltd 13 ©Copyright AARNet Pty Ltd 14 The Story So Far … Design Issues for AARNet3 • Current network (AARNet2) built on Optus ATM has • Redundancy & Resilience been in operation since 1997 • Support for IPv4 and IPv6 • Operates a STM-1 ring to Hawai‘i and Seattle on –unicast and multicast Southern Cross, primarily for research but some commodity via Pacific Wave –line rate performance • Buys commodity access from Optus or Telstra • Traffic Accounting and Monitoring • AARNet Pty Ltd (APL) went to market in February • End to end performance measures 2003 for a new network • Support QoS (diffserv) • Multiple announcements in December 2003 creating • Support for large traffic flows, jumbo frames basis for AARNet3 ©Copyright AARNet Pty Ltd 15 ©Copyright AARNet Pty Ltd 16 Redundancy & Resilience Traffic Accounting and Monitoring • Dual points of presence (POP) in major capital cities • Flow based traffic accounting • Diverse, dual unprotected national links • Scaling issues require function to be moved to the edge of the network –Will use MPLS Fast Reroute for protection • Use anycast addressing so data supplied to a central –Provides ability to burst above capacity collector in an emergency • Use single metro dark fibre pair to connect intra city • Centralise reporting to a POP based server POP sites • Provides AARNet with a window on the network • Creates rings between cities allowing it to identify anomalous traffic, e.g. DDOS • Provides opportunity for clients to build diverse, redundant connections to AARNet ©Copyright AARNet Pty Ltd 17 ©Copyright AARNet Pty Ltd 18 3 Other Issues AARNet3 and SX TransPORT • End to end performance measures –Desire to measure performance from member site –Provide connectivity reports on core services • Support QoS (diffserv) –Need to support VoIP and VideoIP traffic –Possibly introduce scavenger service • Support for large traffic flows, jumbo frames ©Copyright AARNet Pty Ltd 19 ©Copyright AARNet Pty Ltd 20 AARNet3 National Backbone Trans Pacific Transmission • Dark fibre pair on each path of Nextgen Network’s • “SX TransPORT” - Dual STM-64 (10Gbps) national SDH backbone. – Hawai‘i - Manoa and Seattle (Abilene, CA*net 4) – Los Angeles (Abilene, CENIC, CUDI) • Initially using STM-64 (10Gbps) service. – Look to add Mauna Kea to Los Angeles path later • Diverse path between Melbourne, Adelaide and Perth • Dual STM-4 (622Mbps) for commodity Internet from another carrier. Capacity likely to be STM-4 – PAIX Palo Alto (Silicon Valley) (622Mbps). – Los Angeles • Solution for Hobart and Darwin still “work in progress” • Add drop offs to existing STM-1’s (155Mbps) – University of South Pacific, Fiji – Possibly Auckland, New Zealand – Connects to 155Mbps path to Tokyo from Hawai‘i ©Copyright AARNet Pty Ltd 21 ©Copyright AARNet Pty Ltd 22 Services Client Connections • DNS Cache and Secondary Servers • Client provides connection to AARNet3 • Usenet News • Can choose the level of diversity and redundancy • Hexago IPv6 Migration Broker • Diverse connection to each POP • DDoS Detection and Mitigation – Two diverse, independent links, one to each POP – Investigate appliances • Dual connection connecting each POP – Interest in automatic detection and filtering – Two links over same infrastructure to single POP – Locate next to transit (and peering) links – AARNet trunks one link to the second POP though switches • AARNet Mirror • VoIP Gateways • AARNet provided diversity • Traffic Performance Measurement and Availability – Single link to one POP, AARNet provides LAN linking both AARNet POP sites and the member ©Copyright AARNet Pty Ltd 23 ©Copyright AARNet Pty Ltd 24 4 Connections through the GigaPOPs High Bandwidth Client Connections • AARNet supplied and managed edge router • No firewall functionality –clients are responsible for their own security • Interacts with client border router via BGP/MSDP • Edge server used to collect flow information • Modem provides out of band access to APL staff ©Copyright AARNet Pty Ltd 25 ©Copyright AARNet Pty Ltd 26 Equipment Current State of Play - National • Core Router – 40Gbps capable • Brisbane – Redundant power but not CPU –Racks in place, switch and router installed, – Packet over SDH to STM-64 (roadmap to STM-256) awaiting backhaul – Gigabit and 10 Gigabit Ethernet • Sydney • Core Switch – Pure L2 switching –UTS site operational – Fast, Gigabit and 10 Gigabit Ethernet only –Rosebery site, racks in place, switch and router • Edge and POP based “Legacy” routers installed – 3 x Gigabit Ethernet (LAN, POP “A”, POP “B”) • Canberra – 1 x Fast Ethernet dedicated to flow accounting – Capability to handle legacy (slow) interfaces –Racks in place, switch and router installed, awaiting cross-connect ©Copyright AARNet Pty Ltd 27 ©Copyright AARNet Pty Ltd 28 Current State of Play - National Current State of Play - International • Melbourne • Palo Alto (PAIX) –Racks in place, awaiting switch and router –Fully operational (Since March) installation and backhaul • Los Angeles • Adelaide –POP site chosen (Telehouse, Wilshire) –Racks in place, switch and router in place and –Tier 1 transit providers chosen operational, awaiting backhaul –Equipment to be ordered • Perth –Backhaul from SCCN cable station - to be ordered –Racks in place, switch and router in place and operational, backhaul being tested ©Copyright AARNet Pty Ltd 29 ©Copyright AARNet Pty Ltd 30 5 The academic and research Regional network • Network capabilities to regional research and education equal to those in metropolitan areas The regional network • Providing at least 1Gbps capacity from AARNet to regional University sites and Keith Burston Research institutions • Providing dedicated 1Gbps and 10Gbps network connections for advanced research projects ©Copyright AARNet Pty Ltd 31 ©Copyright AARNet Pty Ltd 32 Building the network - the AARNet model The regional network • Combination of different infrastructures –The backbone is a dark fibre pair on the Nextgen national network –Other capacity obtained from working with: • Electricity companies with fibre on power poles • Local AARNet members • State governments • Funding from –DEST as part of national AREN project –CCIF Funding –AARNet members