NETS Special Recognition Award – Spring 2014

A. Criteria Special Recognition Awards are designed to recognize single instances of extraordinary work performed by employees. Employees may qualify for a Special Recognition Award if one or more of the following criteria are met. The employee has: 1. Performed single-instance services that are of outstanding quality or of unusual importance to the execution of UCAR's programs; 2. Displayed commitment to activities or demonstrated outstanding skill or effort above and beyond his or her prescribed duties and workload; 3. Saved significant time or money; and 4. Maintained an excellent level of performance during an organizational emergency or period of high stress, or shown great ingenuity or perseverance. Special Recognition Awards are not for the purpose of recognizing sustained excellence in assigned duties; such service is rewarded through the regular salary administration system. Nor are the awards intended to recognize sustained performance of duties not included in the employee's job description. In those instances, a new job description should be generated and reclassification considered. B. Eligibility All employees are eligible. Either an individual or a team may be nominated. UCAR visitors and former staff are also eligible as long as the work being recognized was done at UCAR. Non-UCAR employees are not eligible for Special Recognition Awards.

 Front Range GigaPop colocation move  The Front Range GigaPoP (FRGP), the Bi-State Optical Network (BiSON), and the Western Regional Network (WRN) are critical wide area networking (WAN) infrastructure for UCAR and all the FRGP, BiSON, and WRN participants in Colorado, Wyoming, California, New Mexico, and Washington. The FRGP and BiSON are managed by UCAR. The Level3 colocation space at 1850 Pearl is the primary hub for the FRGP, BiSON, and WRN WAN services. This criticality and breadth of impact made this project particularly visible and the impact was broad and large.  UCAR as the contracting agent and manager the FRGP and BiSON was notified in mid-January 2014 that the National LambdaRail (NLR) would be ceasing services with less than 30 days notice on 2/17/14. The NETS FRGP team immediately starting working on contingency plans to avoid services shutdown. One of those critical services was the FRGP Level3 colocation services shared in NLR space. In the end, NLR shutdown on 3/17/14, but this was still less than 60 days notice to move critical and complicated services.  Team: The Following is the core team that put in the long hours and hard work on this project. While there were others who made contributions along the way, the NETS management team would like to recognize this core team for all their hard work over the duration of the project. There is a wiki page if you’d like to see more information about the project: https://wiki.ucar.edu/x/iTDYE . Pete Siemsen (lead), Bryan Anderson, Armando Cisneros, Scot Colburn, Susan Guastella, John Hernandez, Carlos Rojas-Torres, . Criteria Met - #1, #2, #4  This team successfully led a multi-component move process under tight timeframes and off-hours to relocate one of the main colocation facilities for the FRGP in Denver at 1850 Pearl Street. The FRGP had sub-contracted space in an NLR suite at 1850 Pearl for approximately 10 years affording the FRGP significant cost savings over this time. Given the NLR shutdown, NETS was required to move out under short notice or face possible power down of all the key gear and connectivity this location provided.  All FRGP equipment and cross connects moved from the existing Level3NLR Suite to new cabinets on a different floor at the Level3 Denver Gateway at 1850 Pearl Street. NLR shutdown was confirmed for 17-Mar-2014 so there was no flexibility in the date or time due to urgency and extreme nature of hard down for this facility.  The project was very successfully completed on 15-Mar-2014.  Elements of this project included: . A case of great teamwork on a very complex project that had to be completed in a very short amount of time. It was completed with great skill, professionalism, communication, and attention to detail on the part of all team members (and also many outside organizations that had be involved and coordinated with including Level3, Internet2, ESnet, NOAA, and FRGP participants). . 1850 Pearl street is a key site for the Bi-State Optical Network (BiSON) and FRGP, including these connections:  FRGP backbone connections to other FRGP routers  BiSON fiber optics circuits for NWAVE (NOAA network)  Internet2 connectivity for all participants  BiSON circuits for BPOP, NCAR, NWSC, CSU, UW, UCB, NWAVE  Member connections at that location including DU, CCCS, CSM, UIS, CTN  Western Regional Network (WRN) lambda to Albuquerque  ESnet peering connection  Netflix cache  CSU Lander research server/tap  And associated services including terminal server, perfsonar machine, environmental monitor, out of band access line, and DYNES equipment. . A detailed and extensive planning and documentation process was undertaken to ensure that all contingencies were accounted for. . This included ensuring that TSIC commodity and Google peering out of 910 15th was up and available for members to accommodate additional load it would be receiving while services at 1850 Pearl were being moved. . The actual work day began at 6am on 15-May-2014 and went for many hours. . Connectivity was restored in a priority fashion as part of the move and the team also worked to minimize equipment downtime by installing spare chassis in new racks a few days in advance and moved the remaining components on the move date.  Other details of project preparation and successful completion included: . Tracking power needs and ensuring the new cabinet locations were prepared and ready in advance. . Tracking roughly 30 different connections in and out of the old location and ensuring the list was current and accurately moved to the new location . Coordination between NETS engineering and infrastructure teams about materials needed including cable management, patch cable inventory/stock availability, working on old rack design and new cabinet configuration and cable management. Many hours were spent preparing materials lists, move procedures, and staging elements . Project follow up included verifying cable dressing, labeling, updating site documentation (rack and colocation connectivity diagrams), and ensuring all access methods were back up and functional in the new location.  Susan was critical to ensuring that all purchases and contracts were in place and in order to enable the work to take place at all. Susan worked with all involved parties, both internal and external, on connection notices, disconnect requests, order and billing tracking, as well as a myriad of other related paperwork required to prepare for and complete this move.  All these activities were above and beyond prescribed duties during an already busy period.  NETS received numerous acknowledgements and gratitude from around the FRGP and UPoP membership for this work, and it was noted repeatedly that good planning minimized the impact for all of the various connected network elements.

In the end, due to efficiencies gained, the cost of the new colocation is no more expensive than the shared NLR colocation, which benefits all of the FRGP. The FRGP has also gained independence from shared colocation, consciously decided on this path, so hopefully, this type of exercise is avoided in the future.