Dark Fibre in the South African Research and Education Network

Simeon Miteff

SANReN Competency Area CSIR Meraka Institute South Africa

September 15, 2014 Who/what is SANReN? Of course, both assumptions were false ;-)

Initial design assumptions - 2007

I Build metro rings with DF from municipalities.

I Interconnect metros with 10G waves from state owned wholesale telco. Initial design assumptions - 2007

I Build metro rings with DF from municipalities.

I Interconnect metros with 10G waves from state owned wholesale telco.

Of course, both assumptions were false ;-) I Compromised on backbone - leased 10G managed circuits.

I Replacing this with dark fibre is the priority!

So what happened?

I Telecoms regulation was relaxed (by accident!).

I Market for metro dark fibre was created (driven by MNOs).

I We procured our metro rings from commercial providers. But... So what happened?

I Telecoms regulation was relaxed (by accident!).

I Market for metro dark fibre was created (driven by MNOs).

I We procured our metro rings from commercial providers. But...

I Compromised on backbone - leased 10G managed circuits.

I Replacing this with dark fibre is the priority! Metro successes

1 I Built metro networks in four largest cities (about 1350km of DF leased for 10 years).

I Deployed access rings, lit with ”grey” 10Gbase-R (small MPLS/Metro-E switches).

I Separate rings lit with DWDM (inter-PoP and large site access) - Cisco ONS 15454 MSTP.

I ROADMs at every node - never touched them!

1Johannesburg, Durban, and Cape Town SANReN Cape Town Metropolitan Network

LEGEND: LINKS 10 GB Link CPUT CPUT Optical 1 GB Link District Six Dept. of Agriculture CPUT Forestry and Fisheries Barc LEGEND: Colours UCT Hiddingh Campus DWDM Ring

Grey Ring UCT Graduate Cape Town school of Business National Backbone CBD Ring SA Museum Main Campus Stellenbosch University Tygerberg Campus SA National Antarctic Programme CPUT National Library Granger Bay To Cape Town Campus Bloemfontein UWC Tygerberg CPUT HSRC (OHC) Sommerset Cape Town square CPUT Tygerberg Campus Medical Research UCT - Groote Council UCT - Sports Schuur Residence Science Institute IS Bree Bellville Distribution Ring CPUT UWC Mowbray UNISA UCT - Rondeberg CPUT UCT - Red Cross Residence Bellville Children's Hospital CPUT Groote Schuur UCT Southern Suburbs Hospital Campus York Gardens Stellenbosch University Ring 2 Business School CHPC Southern Suburbs UCT Ring 1 Rochester House UCT - Mowbray Cottage House

Stellenbosch SKA University Head Office CPUT UCT Groote Schuur Liesbeek Gardens UCT Residence

To Port SAAO CSIR Stellenbosch Elizabeth Cape Town Campus iTemba List of Acronyms Labs CSIR - Council for Scientific and Industrial Research CHPC - Centre for High Performance Computing CPUT - Cape Penninsula University of Technology SAAO - South African Astronomical Observatory SKA - Square Kilometer Array UCT - University of Cape Town UWC - University of the Western Cape Dabbling in long-haul

I 2010: TENET links Durban → Mtunzini (˜140km) to reach SEACOM CLS (DWDM - 2 spans).

I 2011: SANReN CA lights ˜100km of OPGW, Carnarvon → MeerKat (SKA precursor) with 1×ER, 1×ZR span.

I 2013: SANReN CA lights ˜200km backbone link, Johannesburg → Potchefstroom, with 3×ZR spans. Work in progress

I Adding 23 sites into existing DF metro rings. 1 I Building new DF metro rings in four towns . I Interconnecting Johannesburg and Pretoria networks.

SANReN Tshwane Metropolitan Network

NLSA WRC - Main Campus

LEGEND: LINKS 10 GB Link

1 GB Link TUT Kollegehof Residence LEGEND: Colours UNISA - Little DWDM Network Theatre Campus TUT Zaragosapark UP - Medical Residence Grey Network UNISA - Skinner Campus TUT Monitor Existing National Backbone Links Street West Residence New Sites / Route / Access Build UNISA - Skinner Street East LEGEND: Patterns

Route network (leased fibre on fiber provider's (DFA) route) TUT Tempo UP - Oderstepoort Access Build (link from route network into a SANREN Site) Residence North Ring Campus

LEGEND: Symbols TUT TUT Kosmos Pretoria Campus Residence Fibre man-hole

Switch UNISA Sunnyside North Router ARC OVI

UNISA - Sunnyside To Johannesburg Registration Building

UP - Groenkloof Innovation Hub CSIR Campus UP - LC campus Pretoria TUT Zethushof De Villiers Residence UNISA To Durban Main Campus

TUT - Arcadia SA National Campus Bio-informatics Institute

DST

South African Council for Weather Services Geoscience TUT Metro Skinner NRF UP Main Campus TUT Legae Residence CHE List of Acronyms SAEON ARC - IAE ARC - Agricultural research council ARC IAE - ARC Institute for Agricultural Engineering ARC ISCW - ARC Institute for Soil, Climate and Weather ARC OVI - ARC Onderstepoort Veterinary Institute HSRC - Main CHE - Council for Higher Education Campus Waterkloof Air CSIR - Council for Scientific and Industrial Research Force Base DAFF - Department of Agriculture, Forestry and Fisheries ARC - Main DST - Department of Science and Technology Campus HSRC - Human Sciences Research Council South Ring ARC - NRF - National Research Foundation TUT Arts ISCW SAEON - South African Environmental Observation Network Campus TUT - Tswane University of Technology DAFF UNISA - University of South Africa UP - WRC - Water Research Council

TUT Buildings & Estates Pretoria Zoological Gardens Medical Research Council

1East London, Pietermaritzburg, Bloemfontein, Vanderbijlpark Future plans

I Two single span DWDM links for WACS capacity activation:

I Cape Town → WACS CLS in Yserfontein (˜100km). I London Global Switch 1 (WACS endpoint) → Telecity HEX. I Begin procurement of national dark fibre:

I 10G managed circuits are already filling up. I Demand is WACS ↔ CPT ↔ PTA/JNB ↔ DUR ↔ SEACOM I Long haul DF is now available.

Challenges: dark fibre supply quality

Two kinds of suppliers:

I Telco selling DF by exception: The fibre plant division has a more forgiving internal customer, in worst case customer receives similar treatment (poor optical performance, immature procedures and documentation).

I Focussed DF provider: Tends to deliver excellent optical performance, good documentation and procedures (get paid when customer is happy), but worst case: supplier lacks an understanding of networks. Challenges: dark fibre is expensive

Country Area: km2 Density Fibre: €/km/pair/year CZ 79000 143 5001 SE 450000 21 10002 ZA 1220000 41 25203 42004

I ZA did not benefit from fibre overcapacity.

I Long haul seems to support natural monopolies.

1STOKAB 9km point-to-point, base SLA 2Cost model from Josef Vojtˇech’sCEF2010 presentation 3Municipality web site: http://tinyurl.com/lpk5x4f 4Semi-public commercial pricing: http://tinyurl.com/leatwpn Opportunities: SKA

SDP location study

I Power vs. data transport cost trade-off.

I 6 potential sites being evaluated.

I Our team investigated link options. SAT technology selection

I 3 proposed systems.

I Independent evaluation by NMMU.

I Potential for future field trials/test-bed in SA. SKA: simplified data transport

SKA: budgetary costs Opportunities: Dark fibre backbone

I Swaps: competitive selling of the same fibre!

I Co-investment is not impossible... End

I Questions and discussion

I Contact: smiteff@csir.co.za

I Thank you for listening! Attribution

I Network/power diagrams: Bruce Wallace, Square Kilometre Array South Africa.

I Provider logos are trademarked by respective owners.

I KAT7 images: Square Kilometre Array South Africa.

I Awesome SANReN diagrams: Siju Mammen (SANReN).