NORTHERN TIER NETWORK NORTH DAKOTA FY16 North Dakota State University, University of North Dakota and North Dakota Information Technology Department /////////////////////////////////////////////////////INDEX

FY16 ANNUAL REPORT 3

KEY ELEMENTS 4

LOOKING AHEAD–CRITICAL UPGRADES FOR NEXT GENERATION APPLICATIONS 4

OVERVIEW 5

FY16 FINANCIAL SUMMARY 9

21ST CENTURY COLLABORATIVE RESEARCH AND ACADEMICS 10

LOCAL AND GLOBAL IMPACT 14

MILESTONES 15

NTN-ND PARTNERS 18

REFERENCES 19

2 | NORTHERN TIER NETWORK FY16 ANNUAL REPORT The report will provide an update on the operations and maintenance of the Northern Tier Network – North Dakota (NTN-ND) for the fiscal year 2016 aligned to the intent of the network as described in the North Dakota Century Code. Contents of this report include a description of the functions of the network and its service to the research and activities in North Dakota, along with its role in the larger regional, ///////////////////////////////////////////////////// national and global research and education (R&E) network community. This report is intended to inform NTN-ND stakeholders across North Dakota, especially those beneficiaries of the exemplary resources that connection to this global community provides. These include students, faculty and staff located at higher education institutions and Tribal Colleges, and the K-12 community.

An overview of NTN-ND and its role within the larger regional, national and global R&E network environment is followed by a current financial report, examples of 21st century collaborative research and academics, highlights of major milestones in the evolution of the R&E network community, and NTN-ND partners.

EVALUATION PROCEDURES Data gathered and analyzed for this document included a review of current financial records, annual operations, recent R&E network traffic summaries, and current trends in use cases of R&E networks.

LIMITATIONS This report is intended to provide an overview of annual operations and maintenance, financial status and highlighted use cases that offer evidence of current and near-term trends salient to this topic. It does not provide detailed technical information on network operations and maintenance. The list of use cases highlights a short list of examples and is not considered to be exclusive of those experienced by institutions across North Dakota.

ADDITIONAL REFERENCES Additional information on peer networks at the regional and national levels as well as primary partners and collaborators of NTN-ND are listed at the end of this document.

“ The Northern Tier Network, part of a national research network, serves entities within and outside this state. The North Dakota University System may use the Northern Tier Network infrastructure only for the purpose of supporting the research and education missions of the North Dakota University System.

N.D.C.C. § 15-10-45

This document is available in print format online at www.ndsu.edu/vpit/ntn

3 KEY ELEMENTS

• N TN-ND continues to enjoy the engaged support of its three state-level partners: the Information Technology Department (ITD), North Dakota State University (NDSU) and the University of North Dakota (UND), all of whom are collectively responsible for oversight and visioning the future direction of this valuable resource. On behalf of this partnership, NDSU is the fiscal agent and supervises operations and maintenance activities.

• T his state connection to the national and global R&E network community depends upon North Dakota’s state government and education network (STAGEnet) to connect higher public education institutions and Tribal Colleges, and public K-12 schools.

• N TN-ND is operated through contracts with private telecommunications companies and higher education services providers, mirroring a successful model prevalent in state and regional network partnerships across the .

• N TN-ND continues to remain fiscally healthy since the time of its completion in 2009. Annual expenditures are designated for ongoing costs of operations and maintenance on equipment, fiber, services of a Network Operations Center, membership dues and network participation fees, and other miscellaneous operating expenses. No money is used to fund staff of NTN-ND partners ITD, UND or NDSU.

• In 2016, North Dakota and its national counterpart, , celebrated 20 years as partners in the greater U.S. research and education collaborative environment. Since 1996, the Internet2 community has worked to develop and expand the networking capabilities that advance research and education on a global scale. Working together, this community solves common technology challenges and develops innovative solutions in support of their educational, research and community service missions.

LOOKING AHEAD – CRITICAL UPGRADES FOR NEXT GENERATION APPLICATIONS

• T he NTN-ND’s collegial partnership with neighboring regional R&E networks includes visioning for current and future needs of academics and research.

• S ome of the closest neighboring states and regional networks who have completed or are in the process of completing network upgrades to 100G include the Great Plains Network (GPN), /MI, BOREASnet/MN, Montana, Nebraska and South Dakota.

• P artnering and staying in step with neighboring regional and global optical networks serving research and education allow students and faculty at all institutions to advance at the same level and ensures continued partnering on research and academics regardless of where institutions are located.

4 | NORTHERN TIER NETWORK OVERVIEW

ABOUT NTN-ND The Northern Tier Network–North Dakota (NTN-ND) is a joint network effort between the State’s Information Technology Department, NDSU and UND to connect North Dakota to the nation’s research and education network. This high-speed backbone network depends upon STAGEnet for institutional connectivity and is operated through contracts with private telecommunications companies and higher education service providers. NTN–ND is the state-owned segment of the larger regional research and education (R&E) network called the Northern Tier Network Consortium, built collaboratively by states in the upper Great Plains and Pacific Northwest in an effort to provide high-speed network connectivity to research universities.

ABOUT NTNC The Northern Tier Network Consortium (NTNC), consists of 12 member states between and Washington, and Alaska. As indicated in Figure 1, these state networks interoperate to form a larger whole, which serves to function as the governing body for the Northern Tier Network. As built, NTN–ND has a raw capacity of 10 circuits capable of 10 gigabits per second (10 billion bits per second) each. In comparison, advertised residential broadband is typically around 12-50 megabits per second (12-50 million bits per second), with the fastest available advertised speed being approximately 200 megabits per second (200 million bits per second). The high capacity of NTN–ND is used to facilitate collaborations in research, including access to shared computing resources, access and transfer of large data sets and use of high- performance networking applications for shared, real-time access to remote instruments across the country and around the world. This includes not only institutions of higher education, but also federal research labs such as Los Alamos National Lab and Pacific Northwest National Lab. Transport of this data takes place primarily over Internet2, the member-owned research and education network founded by the nation’s leading higher education institutions. NTN–ND enables North Dakota’s institutions of higher education to connect to this national resource.

Alaska CANARIE/BCnet The Northern Tier Network Consortium

Asia Grand Spokane Forks Seattle Coeur Missoula Bismarck d’Alene Fargo Houghton Marquette Pullman Dickinson Superior Sault Ste. Marie Miles City Wahpeton Moscow Yakima Ashland Billings Aberdeen Powers Bozeman Eau Portland Claire Marinette FIG.1 Minneapolis Green Bay Rapid City Sioux Falls Idaho Falls Lansing South- Pocatello Madison field Boise ORION Milwaukee American Falls Iowa Hillsdale Ames Ann Laramie City Stateline Arbor Omaha Lincoln OARnet NTNC Regional Optical Networks Salt Lake Fort Collins BOREAS NTN-ND City FRGP/BiSON PNWGP Boulder IRON SD-REED Denver Merit UNL Kansas Colorado Springs City MUREN Wiscnet Pueblo Internet2 National Backbone 2/19/2016

Figure 1: Full Northern Tier Network Consortium path (2015)

5 The NTNC seeks to develop and sustain advanced networking capabilities and shared cyber infrastructure facilities to support the educational, research and economic vitality of the Northern Tier region. Primary goals of the NTNC include:

• p roviding a premier research network that connects the Northern Tier states from current endpoints in Chicago and Seattle to ensure that every Internet2 member in the Northern Tier has the ability to establish an appropriate high-speed connection to a national or international aggregation point;

• m aximizing network peering and interconnection opportunities between Northern Tier networks and other research networks and networking consortia;

• p roviding a network which can serve as a foundation for regional economic development, as well as academic and research development;

• leveraging intellectual, political and financial resources across the region to expand relationships with state and national leaders, attract greater federal interest, maximize opportunities for grant and contract support and create greater leverage with vendors;

• e nhancing national security and network redundancy by establishing one or more interconnection points between NTNC and Canadian national networks.

REGIONAL PARTNERS Institutions and individuals engaged in the management of NTN-ND represent the North Dakota segment by meeting and communicating regularly with partners at institutions and research and education networks at the national level. These include the NTNC and several regional partners such as the Broadband Optical Research, Education and Sciences Network (BOREAS), the Great Plains Network (GPN), Michigan’s research and education network (Merit), the Northern Lights Gigapop (NLGP), the Pacific Northwest Gigapop (PNWGP) and the University of -Madison. All regional partners are part of the national research and education network, Internet2.

6 | NORTHERN TIER NETWORK INTERNET2 Internet2 is a member-owned advanced technology community founded by the nation’s leading higher education institutions. In addition to its 305 U.S. member universities, Internet2 serves more than 70 government agencies, 42 regional and state education networks, 84 leading corporations within our community and more than 65 national research and education networking partners representing more than 100 countries.

Focused on members’ technology needs since 1996, Internet2 provides a collaborative environment for U.S. research and education organizations to solve common technology challenges and develop innovative solutions in support of their educational, research and community service missions. Internet2 provides a unique set of global capabilities to members for the development of new applications and services specifically designed to meet the needs of U.S. researchers and educators – including a 100 gigabit-per- second network that not only delivers reliable production services for high-performance needs, but also creates a powerful experimental platform for the development of new applications.

The fourth generation of the Internet2 Network has now been deployed, providing an unprecedented 8.8 Terabits of capacity, reaching into underserved areas of the nation. Internet2 Advanced Layer 2 Service allows members to build Layer 2 circuits between endpoints on the Internet2 Network and beyond (Figure 2), providing users with cost-effective, highly reliable solutions for initiatives involving “big data” needs of global science researchers, allowing innovators to deploy new solutions on the network itself to further increase service and capacity of the network.

FIG.2

Figure 2: Internet2 Network Infrastructure Topology (2016)

7 NTN-ND PARTNERS AND RESPONSIBILITIES At the state level, the NTN-ND is the segment of consortia-shared cyber infrastructure that spans across North Dakota and is part of the larger, regional effort of the NTNC whose aim is to provide high-speed network connectivity to research universities in states along the span. NTN-ND is owned and operated by its three in-state partners, the State Information Technology Department (ND-ITD), NDSU and UND. At the time of its inception, initial operational agreements for NTN-ND between the state’s research universities and ITD resulted in establishment of the NTN-ND, a partnership among these three entities. NDSU holds the responsibility to provide administrative and fiscal agent responsibilities for NTN-ND. As one of the main state-level segments of the NTNC, NTN-ND subscribes to the mission and goals of the larger regional research and education network consortia.

STATEWIDE PARTNERS Serving as the administrative and fiscal agent for NTN-ND, NDSU brings together representatives of its research and education stakeholders annually to provide updates related to network use and activities and how these activities impact research and academics for all connected institutions. Stakeholders include representatives of the statewide North Dakota University System and each of the public universities and technical colleges in that system, the Tribal Colleges, K-12 education technology services and ITD.

PART OF THE BIGGER PICTURE ~ REGIONAL ~ NATIONAL ~ GLOBAL The continued expansion and strengthening of the global R&E network fabric is built on the vision and efforts of state and regional R&E networks. The footprint of the current U.S. national backbone (Figure 2) includes a more heavily populated layer of regional- and state-level networks and connectors that serve to solidify the R&E network covering the country as compared to its original build in the mid 1990s (Figure 3). A 2009 federal stimulus-funded grant supported the build of a 100G national backbone intended to connect increased numbers of community-anchor institutions, including schools, libraries, health care facilities and other public institutions to advanced broadband capabilities. As details are gathered to plan for reinvestment in our networks, the leaders responsible for the national backbone are acutely aware of the need to conduct the planning discussion inclusive of the whole community, keeping in mind the critical alignment of national goals with those at the state and regional level.

FIG.3 Figure 3: Internet2 Gigapops (1997)

8 | NORTHERN TIER NETWORK FY16 FINANCIAL SUMMARY

For fiscal year 2016, NTN-ND brought in $1,240,409 of revenue, expended $715,347, and had $346,200 in depreciation expense. Excess revenue from FY16 ($181,800) is designated toward future equipment replacement and acquisition. Since completing the full build for this path in 2009, NTN-ND has remained fiscally healthy. No money is used to fund staff of NTN-ND partners ITD, UND or NDSU during this time period.

These numbers do not reflect a 4.05 percent budget reduction to the state’s general fund as directed by the governor in February of 2016. It is important to note that the 4.05 percent reduction for FY16 was posted to accounts in FY17. A mandate for an additional 2.5 percent cut to budgets in FY17 will further reduce revenue to this fund.

While the current reductions leave NTN-ND with a budget that is adequate to cover ongoing expenses for annual operations and maintenance, further cuts will reduce the ability by NTN-ND to defer funds toward replacement and upgrades of equipment that have been in place since the original build of the network in 2007-2009. A delay or abandonment of upgrades to North Dakota’s segment of the Northern Tier Network span that are critical to continued peering with adjoining networks in the region would create a bottleneck or a complete disconnect in the network, obstructing access to R&E resources by the research and education community across the partners of the NTNC, along with institutions in North Dakota.

All NTNC member states are in the process of completing upgrades to their state networks to ensure local universities and research centers continued access to bandwidth critical to their daily functions in research and academics. A few examples of this evolution is evidenced by those states and networks closest to and bordering the NTN-ND segment. These include the Great Plains Network (GPN), Merit Network/MI, BOREASnet/MN, Montana, Nebraska and South Dakota. Other NTNC members who have also completed upgrades in recent years include Iowa, Nebraska, Washington and Wisconsin.

In each case a network upgrade for an NTN-ND neighbor carries the expectation that collaboration with the neighboring NTN-ND segment will continue, ensuring redundancy and back-up paths to Internet2 connector locations are in place across the network. Similarly, NTN-ND holds the same expectation for others in return. The potential inability of any single state or network segment to provide the fiber, hardware and software capacity that ensures interoperability across the multiple networks results in gaps in service to the region and most critically, to the local institutions we all serve.

$22,437 3% REIMBURSEMENT $197,781 $138,393 $305,409 28% 19%

FY16 FY16 REVENUE EXPENSES

$89,407 REVENUE 13% $267,329 $935,000 37%

EQUIPMENT RELATED COSTS INTERNET2/NTN FIBER RELATED COSTS OTHER OPERATING NOC

9 NTN-ND ANNUAL EXPENDITURES OCCUR IN FIVE MAIN CATEGORIES: EQUIPMENT INTERNET2/NTNC DUES AND PARTICIPATION FEES Costs related to equipment acquisition and Annual Internet2 participation and connectivity maintenance fees. fees, NTNC membership dues, sponsored participant fees and costs for Internet2 to accept and route traffic to and from NTN-ND. Internet2 FIBER membership dues and fees continue to change as Operations and maintenance fees on fiber, and the organizational structure for Internet2 and their rack and power required to house equipment in member community evolves to address the needs space provided by the vendor. of current research and academics.

NETWORK OPERATIONS CENTER (NOC) OTHER OPERATING Management of network equipment, including Includes travel expenses for NTN-related meetings troubleshooting and response coordination for by NTN-ND partners, insurance, minor supplies, unplanned outages. printing and shipping costs.

21ST CENTURY COLLABORATIVE RESEARCH AND ACADEMICS

GENOMIC SEQUENCING IS EXPANDING Genomics research is rapidly becoming one of the leading generators of Big Data science, with the potential to equal if not surpass the data output of the high-energy physics community. Like physicists, university- based life-science researchers must collaborate with counterparts and access data repositories across the nation and around the globe.6

TRANSFERRING GENOMIC DATA… In 2012, live demonstrations of ultra-high-speed data exchanges between three world-class genomics institutions successfully confirmed genomic data transfer at a sustained rate of almost 10 gigabits per second (Gbps) over a link connecting US and China R&E networks. The data rate is equivalent to moving more than 100 million megabytes – over 5,400 full Blu-ray discs – in a single day.

As part of that live demonstration, 24 gigabytes of genomic data were transferred from Beijing to University of , Davis via the 100G Internet2 network in 30 seconds. A file of the same size sent via the public took more than 26 hours. In contrast, transporting this data via a hard drive by airplane would take 17 hours.2 “ Genomics has revolutionized the life sciences. While the cost of DNA sequencing is steadily decreasing, the amount of data generated with next-generation sequencing (NGS) technologies is growing at an unprecedented pace. In the age of “Big Genomics Data,” how to conveniently share the tremendous volume of data has become a significant research bottleneck.2

10 | NORTHERN TIER NETWORK ESNET13 The (ESnet) is the Department of Energy’s (DoE) high-performance networking facility, engineered and optimized for large-scale science.

ESnet was founded in 1986, soon after the creation of the global Internet. Since 1990, ESnet’s traffic has increased by a factor of 10 every 48 months, roughly double the growth rate of the commercial Internet (Figure 4). ESnet serves more than 40 DoE sites, connecting them to over 100 research and commercial networks worldwide. The richness of ESnet’s global connectivity is motivated by the fact that 80 percent of its traffic originates or terminates outside the national laboratory complex. This pattern in turn reflects the collaborative, increasingly international nature of scientific research.

ESnet is a user facility designed to overcome the constraints of geography. ESnet succeeds at this task by offering unique capabilities, and optimizing the facility for data acquisition, data placement, data sharing and data mobility building networking capacity to support research and education. “ Discovery is unconstrained by geography.

Inder Monga, Deputy of Technology, SND, CTO, ESnet13

FIG.4

Figure 4: History of ESnet’s research science data transfer rates from 1994 to 2014

11 XSEDENET: AN ADVANCED NETWORK ADVANCING SCIENCE IN THE AGE OF BIG DATA14 To support more than 8,000 global researchers engaged in compute-intensive team science disciplines such as energy, genomics, climatology and physics, XSEDE provides a scalable, resilient high-speed networking infrastructure capable of transporting massive datasets and delivering innovative applications and cloud services. Although the enhanced XSEDENet is one of the most advanced networks available today, continuous improvements are needed to stay ahead of the scientific big data curve. Internet2 and XSEDE provide a platform to further improve performance, reliability and predictability for distributed science applications. (Figure 5)

FIG.5

Figure 5: Internet2 and XSEDE research platform

12 | NORTHERN TIER NETWORK INVESTING IN CYBERINFRASTRUCTURE: NORTH DAKOTA, AN EPSCOR STATE SINCE 1985 Completed in 2012, the Experimental Program to Stimulate Competitive Research (EPSCoR) 2030 national plan3 includes critical recommendations for both EPSCoR states and the federal and state agencies that fund their activities. “ I think the states need a strategic plan for cyberinfrastructure because it is the most enabling piece of infrastructure that supports all kinds of science. Every science is enabled by cyberinfrastructure.

Gwen Jacobs, Director of Cyberinfrastructure University of Hawaii System

CLOSE THE CYBER INFRASTRUCTURE GAP3 A significant investment continues to be needed to close the “cyber gap” in EPSCoR states. The costs of developing cyberinfrastructure-related network connections are generally higher in EPSCoR states because of distances from available Broadband connections. This gap places EPSCoR states at a disadvantage when competing for federal funding which requires a well-developed cyberinfrastructure. It also places EPSCoR states at a disadvantage (1) when trying to attract nationally competitive researchers whose work contributes to the nation’s competitiveness and innovation goals, (2) when connecting research collaborators who are located on other campuses or in other states and (3) when attracting high-tech companies that depend on cyberinfrastructure and competitive faculty who conduct computational research.

The EPSCoR Research Infrastructure Program has enabled many state universities and colleges to develop their research potential and capacity through investments in new faculty, research instrumentation and core facilities, student research opportunities and outreach to citizens within their states. The tools and infrastructure necessary to remain competitive in this new arena of computational and data intensive approaches are changing rapidly, requiring major investments in cyberinfrastructure to meet growing needs in computation, data management, visualization and high-speed, high-bandwidth networking.

When EPSCoR institutions lag behind this trend, they find that their faculty and students spend valuable research time maintaining their own systems, and struggle to maintain the expertise required to use the technology effectively.

The cyberinfrastructure landscape has changed so rapidly that even the top-tier states and institutions are having a hard time staying on top of community standards in networking, computational platforms, collaboration environments, identity management and security, and keeping up with new software architectures, etc. Even for institutions that have invested heavily in this area, it continues to be a steep learning and price curve. EPSCoR has led the way in several areas of the nation with significant cyberinfrastructure now penetrating the Northern Tier of states (from North Dakota through Montana, and Idaho to Washington), the Northeast (Vermont, Maine, New Hampshire, and Delaware) and Louisiana. Yet work remains with many EPSCoR institutions needing basic core cyberinfrastructure. Without strategic action, EPSCoR states stand to lose their investments in intellectual capital and research capacity, as the “best and brightest” scientists are recruited by institutions that are making cyberinfrastructure investments that can support their research.

13 LOCAL AND GLOBAL IMPACT

//////////////////////////////////////////////// //////////////////////////////////////////////// Investment in local and regional infrastructure positions institutions to obtain more research and development funding, positively impacting local economies at even higher rates.

The U.S. and invest $453 billion annually in research and development.1 $453 BILLION

In 2013, the National Science Foundation provided more than $5.3 billion in funding to colleges, universities and academic consortia.8 $5,300,000,000

/////////////////////////////////////////// K12 EDUCATION /////////////////////////////////////////// Alignment to US Government’s 2016 National Ed Tech Plan12

Learning, teaching and assessment enabled by technology require a robust infrastructure. The NETP16 goal of Infrastructure states that all students and educators will have access to a robust and comprehensive infrastructure when and where they need it for learning. Key elements of this infrastructure include ubiquitous connectivity, powerful learning devices, high-quality digital learning content and Responsible Use Policies (RUPs).

//////////////////////////////////////// GLOBAL RESEARCH //////////////////////////////////////// Sharing data around the world of all scientific papers now Global research and development have co-authors from two or budgets have more than doubled in more countries.4 the past 15 years.4 15 25% YEARS

5 //////// ///////

By year end 2016 By year end 2017 By year end 2018 By 2020

Connected car subsystems will The number of connected 20 percent of business Internet of Things (IoT) – more exceed 600 million units. health and fitness devices will content will be authored than 35 billion things will be exceed 580 million units. by machines. connected to the Internet.

6 billion connected things 47 percent of devices will have will be requesting support. the necessary intelligence to request support (think vehicle 3 million workers globally will engines, connected prostheses, be supervised by a “robo-boss” vending machines, vacuum cleaners, printers, air fresheners, 20 percent of smart security cameras, parking buildings will have suffered meters, soap dispensers and 14 | NORTHERN TIER NETWORK from digital vandalism. aircraft). NTN-ND AS PART OF MILESTONES NATIONAL R&E HISTORY

1960s - 1970s Initial research and development of theory is explored. The U.S. Department of Defense’s Advanced Research Projects Agency (DARPA) collaborates in deployment of ARPAnet, the first wide-area network.

1980s The education community expands the Internet to provide networking services to computer science researchers. This was the first expansion of the Internet beyond the group of early Department of Defense funded projects. “when the Internet was commercialized everything went to industry, our research backbone had dissolved…” Internet2 (2016)

1996 Founding of Internet2 high-performance research and education (R&E) network is established through a partnership of 34 universities. States in the ‘northern tier’ begin discussion on connecting to this new high-speed backbone. NDSU and UND become Internet2 members, sponsoring the North Dakota University System and the nine public institutions and K-12. The state’s Tribal Colleges joined the group of sponsored participants in 2013.

2003 The NTNC is established by higher education representatives from nine states across the upper Great Plains and northwestern U.S.

2005 North Dakota launches efforts to deploy its own state segment of the NTNC span through initial planning grant funds.

2006 More than 35,000 route-miles of dark fiber are held by U.S. research universities through the Abilene (Internet2) network.

An initial planning grant from National Science Foundation and one-time federal funds are secured to begin work on Northern Tier Network – North Dakota (NTN-ND).

2007 The North Dakota Legislature formally endorses the NTN-ND project through a $2.73 million one-time appropriation.

CONTINUED ON NEXT PAGE 15 2009 NTN-ND goes live in April, marking the completion of the network segment build spanning the state and connecting to NTNC paths to the east and west.

NTN-ND hosts the annual NTNC meeting in Fargo and celebrates the lighting of the NTN-ND segment with a dedication ceremony. “This collective effort that has helped create the new Northern Tier Network fills an absolutely critical gap in our nation’s cyberinfrastructure.” Doug Van Houweling, President and CEO | Internet2 (2009)

2010 – CURRENT NTN-ND moves into full operations, turning its focus toward building and strengthening partnerships.

NTN-ND and regional partners collaborate to complete federal grant awards that further expand the capacity of the national R&E network. • N ational Science Foundation (NSF) - North Dakota/South Dakota Interconnect: connects the NTN-SD segment to NTN-ND at Fargo • A merican Recovery and Reinvestment Act: Expands the national R&E backbone to initiate the Internet2 U.S. Unified Community Anchor Network (UCAN) • N SF / EPSCoR: ND Tribal College Cyber Connectivity (C-2) initiative improves campus cyberinfrastructure, enabling connection to Internet2 by all Tribal Colleges in North Dakota. • N SF: NTN-ND and the Pacific Northwest Gigapop build the Northern Wave - a new optical data connection between Chicago and Seattle.

“Northern Wave brings a significant new capacity to the network through improved communication facilities, as well as easy exchange of data for initiating collaborations with other institutions. This is especially important in the establishment of large competitive research centers.” Kalpana Katti, NDSU Distinguished Professor of civil engineering (2015)

National Oceanic and Atmospheric Administration (NOAA) partners with NTNC and Pacific Northwest Gigapop to use NTNC’s Northern Wave optical data connection as a backup network pathway.

“Much of N-Wave’s network foundation is based in its partnerships with the research and education community…NOAA’s partnership with the NTNC extends the overall stability of N-Wave and sets the foundation for future scientific collaboration.” Robert Sears, NOAA network engineer (2013)

“Access to NOAA resources is crucial for my work as a researcher and state climatologist. Without access to NOAA resources, there would not be any state or regional climatology data, and there would not be any historical context for today’s weather.” Adnan Akyuz, NDSU Assistant Professor Director of the North Dakota Agricultural Weather Network (2013)

Northern Plains UAS is named one of six Test Site Centers of Excellence and FAA Test Sites, providing an avenue for the industry to expand advanced UAS research.11

Positive impacts / successes of the burgeoning UAS industry include: • N orthrup Grumman and General Atomics are the first tenants at Grand Sky business park, Grand Forks, ND. • I nternational companies are expanding into North Dakota with physical presences or significant research projects. • N orth Dakota is witnessing a growing cluster of private sector companies centered on this industry.

“A large-scale UAS operation allows more agricultural data to be collected faster and at altitudes up to 8,000 feet…we’re making important strides in the UAS and precision ag sectors in North Dakota, as well as supplying data to the Federal Aviation Administration in an effort to allow commercial UAS to operate simultaneously with manned aircraft in the future.”7 Drew Wrigley, ND Lt. Governor (2016)

“The expertise residing at our UAS test site and universities continues to attract interest from companies around the world.” Brian Opp, Manager, Aerospace Business Development, ND Department of Commerce (2015)

16 | NORTHERN TIER NETWORK 2015 NDSU campus technology network infrastructure is extended to USDA Biosciences Research Laboratory to support partnerships with NDSU faculty and student researchers.

“Extension of the NDSU network into USDA facilities will expand opportunities for collaborative research, and will likewise enhance the ability of NDSU and USDA collaborators to attract jointly submitted competitive grants.” 9 William Kemp, agricultural administrator at the Red River Valley Agricultural Research Center (2015)

Fargo is named as the location for the new National Agricultural Genotyping Center, expanding opportunities for research activities with NDSU faculty, staff and students.

“NDSU is in a unique position to extend existing high-performance network resources to connect researchers with a broad range of bandwidth-intensive resources they need to advance their work” 10 Terry Wieland, Director | Network Engineering and Operations, NDSU Information Technology (2016)

“We are excited for future collaboration with NDSU students, faculty and staff…We are offering an internship for students and look forward to providing meaningful experiences for students to prepare them for full-time positions in their science-related fields.”10 Megan Palmer, Manager, NAGC (2016)

2016 Internet2 Network Carries Record High Traffic in April Milestone represents significant achievement by Research and Higher Education Community

On May 16, 2016 Internet2, operator of the nation’s largest and fastest, coast-to-coast R&E infrastructure, announced that it carried 100 petabytes of data across its backbone in April, up almost 50 percent since last year. The record high traffic is due to an uptick in traffic from rapidly expanding collaborations among researchers and educators around the globe who use Internet2 to accelerate their work. Growth in genomics, physics and other big-data applications, as well as the increased use of video on campuses and the continued adoption of 100G and cloud services have all contributed. Since February, 2008, traffic carried on the Internet2 backbone has increased 2,500%.

Built by and for the research and education community, the Internet2 Network offers 17.6 Terabits of optical capacity and 100 gigabit Ethernet technology across its entire footprint.

Internet2 and NTNC celebrate 20 years as partners in the greater U.S. R&E collaborative environment working together to solve common technology challenges and develop innovative solutions in support of their educational, research and community service missions.

NTNC now represents higher education, state and regional networks and research entities across 12 states.

Compared to its early years, Internet2 now operates the nation’s largest and fastest coast-to-coast R&E network, serving more than 90,000 community anchor institutions, 317 U.S. universities, 64 affiliate and federal affiliate members, 43 regional and state education networks, 81 leading corporations working with our community and more than 65 national research and education networking partners representing more than 100 countries.6

17 A variety of state, regional and national colleagues partner with NTN-ND to contribute to NTN-ND PARTNERS the collaborative environment where US research and education organizations solve common technology challenges and develop innovative solutions in support of their educational, research and community service missions.

PARTNER WEBSITE RELATIONSHIP

BOREAS is a Regional Optical Network (RON) providing services to the advanced production and experimental network requirements of the research and education institutions in this region. BOREAS is a collaboration of Broadband Optical Research, Education boreas.net four major research institutions in the upper Midwest: Iowa and Sciences Network (BOREAS) State University, the University of Iowa, the University of Minnesota, and the University of Wisconsin-Madison. BOREAS members are part of the NTNC.

Infinera www.infinera.com Vendor for optical networking hardware.

The primary provider and operator of the nation’s research Internet2 www.internet2.edu and education network.

Midcontinent Communications www.midcocomm.com Provides fiber between Grand Forks, N.D., and South Dakota.

North Dakota Information Technology The statewide provides gateway services Department / Statewide Technology www.nd.gov/itd/ to the public internet and to Internet2 for designated sites. STAGEnet is operated by the State Information Technology Access for Government and www.stagenet.nd.gov Department, which is one of three NTN-ND partners. ITD is Education Network (STAGEnet) a member of the NTNC.

One of three NTN-ND partners, that along with the University of North Dakota, has been an Internet2 member for North Dakota since 1996. Together NDSU and UND provide North Dakota State University (NDSU) www.ndsu.edu Internet2 sponsorship for the other nine NDUS institutions, the ND Tribal Colleges and the public K-12 schools. NDSU is a member of the NTNC.

Northern Lights Gigapop, www.northernlights.gigapop.net Connects NTN-ND to Internet2. The University of Minnesota University of Minnesota www.umn.edu is a member of the NTNC.

Northern Tier Network Consortium The NTNC consists of member states that represent the www.ntnc.org collective interests of their institutions in activities related (NTNC) to networking and network infrastructure.

Interconnect hub in Seattle, Wash., providing connections between the Northern Wave and other national and Pacific Northwest Gigapop (PNWGP) www.pnwgp.net international R&E networks. PNWGP and the state of Washington are members of the NTNC.

18 | NORTHERN TIER NETWORK “ The technologies developed and leveraged by this community not only advance R&E, but accelerate discoveries across the globe and have lasting impacts on the quality of human life.

Internet2 (2016)

PARTNER WEBSITE RELATIONSHIP

www.sdstate.edu/informa- South Dakota High Speed Research, Provides high speed connectivity for six South Dakota tion-technology/university-net- Education and Economic Development public universities to Internet2, regional networks and working-and-research-computing interconnect sites including the NTN, GPN, the Northern Network (SD REED) / South Dakota Lights Gigapop, and ESnet. Several institutions and Board of Regents www.sdbor.edu research centers in the state are members of the NTNC.

Partners in the NTN, the University of Montana became an Internet2 Network connector site in 2015, connecting to the www.umt.edu University of Montana / Internet2 network with a 100G fiber connection in Missoula and manages operations of the NTN segment for Montana. Montana State University www.montana.edu UMT connects a portion of Montana’s higher education institutions and tribal colleges, as well as the U.S. Forest Service. Both universities are members of the NTNC.

One of three NTN-ND partners that along with NDSU, has been an Internet2 member for North Dakota since 1996. Together UND and NDSU provide Internet2 sponsorship for University of North Dakota (UND) www.und.edu the other nine NDUS institutions, the ND Tribal Colleges and the public K-12 schools. Member of the NTNC. UND is a member of NTNC.

Provides network operations center (NOC) services for University of Wisconsin, Madison www.wisc.edu NTN-ND and is a member of the NTNC.

REFERENCES/ CITATIONS

1) Bateman, P. (2014). Research is now a global game. Times Higher Education. Available online 8) National Science Foundation (2014). FY15 Budget request to congress. Available online www. www.timeshighereducation.com/comment/opinion/research-is-now-a-global-game/2013292.article nsf.gov/pubs/2014/nsf14041/nsf14041.pdf 2) BGI, (2012). BGI demonstrated genomic data transfer at nearly 10 gigabits per second between 9) North Dakota State University (2015). NDSU extends network resources to USDA research US and China. Available online www.genomics.cn/en/news/show_news?nid=99118 center. NDSU News. Available online https://www.ndsu.edu/news/view/detail/22709/ 3) Experimental Program to Stimulate Competitive Research (2012). EPSCoR 2030 10) North Dakota State University (2016). National Agricultural Genotyping Center to benefit from Workshop report. National Science Foundation. Available online www.nsf.gov/od/oia/programs/ NDSU network. NDSU News. Available online https://www.ndsu.edu/news/view/detail/24717/ epscor/2030%20Report.pdf 11) U.S. Federal Aviation Administration (2013). Center of Excellence and FAA Test Sites. 4) Fischer, K. (2014). U.S. Seen as weak on global research collaboration. The Chronicle of Available online www.faa.gov/uas/programs_partnerships/coe_test_sites Higher Education. Available online www.nytimes.com/2014/07/21/us/us-seen-as-weak-on-global- www.business.nd.gov/aviation/NorthernPlainsUAS research-collaboration.html?_r=0 12) U.S. Department of Education (2016). 2016 National education technology plan. Future ready 5) Gartner, (2015). Strategic predictions for 2016 and beyond: The Future is a digital thing. learning: Reimagining the role of technology in education. Available online https://tech.ed.gov/ Gartner, Inc. Available online www.gartner.com and http://www.gartner.com/newsroom/ files/2015/12/NETP16.pdf id/3143718 13) U.S. Department of Energy, (2015). ESnet Energy Sciences Network. Available online www. 6) Internet2, (2015). Research solutions: Accelerating genomic research with advanced networking es.net collaborations. Internet2 Research Solutions. Available online www.internet2.edu/media/ 14) XSEDE (2016). Extreme Science and Engineering Discovery Environment. National Science medialibrary/2015/04/06/clemson-ncbi--accelerating-genomics-research.pdf Foundation. Available online www.xsede.org. 7) Jewett, B. (2016). ND leaders visit Israel for trade mission, meet with manor unmanned aircraft manufacturer. Grand Forks Herald. Available online http://www.grandforksherald.com/news/ business/3936988-nd-leaders-visit-israel-trade-mission-meet-major-unmanned-aircraft 19