Russian International Connectivity - Slava

International Committee for Future Accelerators (ICFA) Standing Committee on Inter-Regional Connectivity (SCIC) Chairperson: Professor Harvey Newman, Caltech

ICFA SCIC Report on the Digital Divide in Russia

Prepared by the ICFA SCIC Digital Divide Working Group On behalf of the Working Group: Slava Ilyin [email protected] .ru 四月 9, 2018

The Digital Divide and Connectivity for HEP in Russia

8 February 2003 V.A. Ilyin, [email protected] Edited by H. Newman

Russian physicists participate in many large-scale HEP experiments, e.g. D0 at Fermilab, Belle at KEK, Zeus at DESY and others. Russian HEP institutes significantly contribute to the construction of all four LHC experiments at CERN, and participate actively in the design of the TESLA experiment at DESY. They have a strong desire to be heavily involved in the LHC data processing and analysis. This involvement should preferably include participation in the development and testing of particle physics Grids. In order to work on Grids and on the data analysis, the quality and quantity of the networking, both internal and international, must be improved. The most important point, when one speaks about the digital divide in Russia, is the weakness of State policy in this field. Only in 1996 the first National State Programme for development of connectivity for science and higher education has been started. Thanks to this Programme the Russian backbone network (RBNet) was created to serve needs of science and higher education. One can state that during last five years there has been a good start towards the formation of a Russian National Research and Education Network (NREN), although we cannot say that the creation process has been completed until the network is able to meet the needs of current scientific and education activities.

One main consequence of this weakness (even absence) of State policy is the chronic under-financing of network programmes. The most recent event, characteristic of this general situation, occurred in December 2002 when funding for the State Programme for the “Creation of a National Computer Network for a New Generation for Science” (in the following we denote this – State Programme CNCN) was removed completely from the State Budget for 2003. As a result, the development of research and education networks in Russia became chaotic. Indeed, significant contributions have been coming from the local budgets of regions and internal financial sources of Russian institutions. There have also been substantial contributions from our foreign collaborators, and international funds (e.g. INTAS, http://www.intas.be, and NATO) have been significant in the case of some “cooperative” and regional networks (a characteristic example is the contribution of DESY to the creation and development of a cooperative network for HENP in Russia). These circumstances, and the recent lack of a coherent national funding programme, have been major contributing factors in deepening the Digital Divide among regions and institutions in different parts of the country. To complete this short introduction to the Digital Divide (DD) report one should state, nevertheless, that the achievements of last five years within the State Programme CNCN has been a good foundation for the next stage of the creating/developing the Russian NREN. In this report the following aspects of the Digital Divide in Russia are presented: 1. general characteristics of the Digital Divide; 2. geographical remarks; 3. telecommunications operators; 4. State Programme “Creation of National Computer Network of New Generation for Science”; 5. backbone networks for science and education;

2 6. international links; 7. cooperative networks; 8. Digital Divide for HEP institutes; 9. maps, tables, schemas; 10. Appendix: ICFA SCIC Questionaire – Responses by Russian Sites.

3 1. General Characteristics of the Digital Divide in Russia Today the capacity of backbone channels for science in Russia is at the level of 30-45 Mbps, although in many cases it is just a few Mbps. At the same time Gigabit/sec networking is coming – there are several 1 Gbps links in Moscow, and some will start in other regions. However, this advanced level is available, if at all, only in the “Last Mile” connections, for financial reasons. For the next 2-3 years there are plans to increase the capacity of the basic links for Russian science by a factor 4-6, to reach the level of 155 Mbps (and in some cases higher). However, when viewed across the whole country, the Last Mile connection speed is characterized by a much wider distribution, heavily weighted on the low side (towards low bandwidths). While a few major scientific centres are connected at 45 Mbps or higher, many other important scientific centres have connectivity that is only at the level of a few hundreds of Kbps (for example, the St. Petersburg Institute of Nuclear Physics). In many cases, regional universities and scientific institutions have even lower connectivity (at the level of tens of Kbps) because they have to buy access to the commodity Internet at monopolistic prices. A characteristic example is the University of Veliky Novgorod. International connectivity is now at the 155 Mbps level. There are three links with this bandwidth in operation now, as discussed later in this report. In recent years three new operators (with fiber-optic infrastructure crossing the country) have started activity in Russia, in addition to the ROSTELECOM state-sponsored (former) monopoly company. One could have hoped that the appearance of new operators would result in lower prices. In practice, however, all operators have tended toward increased prices, in sharp contrast to the trends in many other parts of the world. To conclude this section, one should stress that there is a Digital Divide of a factor of order 100 between networks in Russia and those in the most advanced countries (the US, Canada, European Union, Japan). One can state that this Divide factor did not change significantly since 1995. The direct cause of this absolutely unnatural situation for Russian science is poor budgets, and in some cases the complete absence of a budget. There are no technical problems, nor problems with manpower and the organization of the corresponding services, that bar the way to having a Gigabit/sec Russian NREN – even today. The current state of development of an NREN in Russia, and its limited progress so far, are the result of the absence of a well thought-out State policy in this field.

2. Geographical Remarks The Russian territory extends across Europe and Asia. The distance from the West end (Kaliningrad) to the East end (Vladivostok) is more than nine thousand kilometres. The geographic boundary between the European and Asian parts of Russia lies in the Ural mountain region (about 1.5 thousand kilometres to the East from Moscow),see the map of Russia below. If we speak about scientific centres, one can divide Russia into the following regions (which however do not follow closely the political subdivisions of the country). They are  Moscow and the Moscow Region (Dubna, Protvino, Troitsk, Puschino, Obninsk, Chernogolovka and others)  The Central region (Tver, Yaroslavl, Tula, Tambov, Voronezh, Vladimir Kursk, Belgorod, Orel and others)1  The Northwest Region (St-Petersburg, Petrozavodzk, Kaliningrad, Novgorod, Vologda and other)

1 Although Moscow is geographically in this region the capital and sites around it should be considered separately from DD point of view. 4  The Southern Region (Rostov-on-Don, Krasnodar, Stavropol, Makhachkala and others)  The Volga Region (Samara, Nizhny Novgorod, Sarov, Volgograd, Kazan, Ulyanovsk, Saratov and others)  The Urals Region (Ekaterinburg, Ufa, Chelyabinsk, Orenburg, Perm, Izhevsk, and others)  Siberia (Novosibirsk, Irkutsk, Kemerovo, Tomsk, Omsk, Krasnoyarsk, Yakutsk and others)  The Far East Region (Khabarovsk, Vladivostok and others).

Universities as well as scientific institutions (laboratories) are located in all of these regions, and in nearly all of the cities mentioned.

3. Telecommunication Operators Historically the first major operator in Russia was ROSTELECOM, which held a monopoly in this field. This company was created as a commercial “daughter” of the State Ministry for Communications. In the last 2-3 years new operators have started at the national level: TRANSTELECOM (a “daughter” of the State Ministry of Transports), GASTELECOM (daughter of the GAZPROM Company) and the Russian Unified Electrical Company (RAO ES is the acronym taken from the Russian). As a result, there is now a choice between different operators, e.g. through calls for tender. However, in many regions and for many institutions there is only one operator available, often with a monopolistic price policy. Moreover, some new operators have begun to increase their prices, shortly after entering the competitive market, approaching the price levels of the former ROSTELECOM monopoly. These trends are opposite to the general worldwide tendency, towards rapidly decreasing prices in telecommunications markets. These four operators have developed fiber optic infrastructure on a national scale. Today the major operator for science in Russia is TRANSTELECOM (see the map of this network below). In many regions of the country there are local operators which play an important role in establishing Last Mile connectivity. The leading example is Moscow, where there are fiberoptic cables of the network of Russian Academy of Science (RAS). The North Moscow Backbone (NMBB, http://www.ripn.net:8080/nmbb/general.html ) also serves scientific institutions in the north and northwest parts of Moscow, and there is a fiberoptic infrastructure called the Southern Moscow Backbone (SMB, http://www.umos.ru ). The central Internet exchange point in Moscow is M9-IX, where all networks are present (see, e.g. http://www.ripn.net:8082/ix/msk/en/documents.html). There are some foreign operators, e.g. SONERA and TELIA, but they are limited to to the provision of international links in most cases. In addition to fiberoptic infrastructure, satellite links also have played play an important role in Russia. This was especially the case in the early 1990’s. Now, due to the huge distances between sites these, links still are important, because they are the sole means of accessing the Internet in several of the more remote regions. Today there are several companies that operate satellite links, the largest one being the State Satellite Communication Company (SSCC, http://www.rscc.net).

4. State Programme “Creation of National Computer Network of New Generation for Science” As has already been said the predecessor of this State Programme, which began in 1996 and finished in 2001, resulted in the creation of the first Russian national backbone network for science and higher education. Another

5 result was the initiation of a set of international links that served science on a national scale. These achievements have been the real steps towards a Russian NREN. The coordination of this Programme has been performed by the Ministry of Industry, Science and Technologies (MoIST), operated now through the Joint SuperComputer Center in Moscow (JSCC, http://www.jscc.ru). A Political Board and an Executive Board were organized to manage this Programme. A new Programme, the “Creation of National Computer Network of New Generation for Science”, started in 2002, and is planned to operate until 2006. This Programme aims to achieve Gigabit/sec bandwidths across the national backbone links, as well as international links by 2006. There are a few other national or cooperative programmes supporting networking for science and higher education in Russia. In the past, the International Science Foundation (the so-called Soros Foundation) had a network program, and one of the first fiberoptic backbones was created in Moscow with its help. A new Federal Program called “Electronic Russia 2002-2010” (http://www.e-russia.ru, with only Russian web-pages), has been started with the main goal being to improve the general networking situation. Note that one of the goals of the State Programme CNCN is to move away from the use of the commodity Internet traffic to the serve science through programmes such as “Electronic Russia”, where the CNCN will focus only on providing support for scientific network traffic. The budget of the State Programme CNCN in 2002 was about 3M US dollars (note that approximately the same figure in total was used by science and education networks from other sources; mainly from regional budgets and the internal budgets of the individual institutions). As already mentioned, the budget of this Programme has been cancelled for 2003 by the Russian Parliament. This event has put the Russian NREN in a situation close to complete collapse. On December 15, 2002 there was a meeting organized at the highest official level, where top- level representatives from major Ministries (MoIST, the Ministry of Atomic Energy, the Ministry of Education, the Russian Academy of Science and others) discussed the problem. It was decided that the necessary budget will be collected by these Ministries, from the Russian Foundation for Basic Research, and the major scientific centres (Moscow State University, Joint Institute for Nuclear Research and RRC “Kurchatov Institute”). This process is not completed yet, but the unofficial news is that the collection of the necessary budget is proceeding successfully. There is a plan to get a new Agreement between the interested (and financed) organizations and institutions, and to develop the Russian NREN within the context of this Agreement. This could also mean that changes in the organizational structures will be made.

5. Backbone Networks for Science and Education

Two main national networks in Russia currently serve scientific research and higher education. They are RBNet (Russian Backbone Network, http://www.ripn.net:8082/rbnet/en/, see below the map of this network), with RIPN (Russian Institute for Public Networks) being the institution responsible for this project, and RUNNet (Federal University Computer Network, http://www.runnet.ru/vtc/index_.htm, see below the map of the RUNNet satellite channels). The former is supported by the Russian Ministry of Industry, Science and Technologies (MoIST), and latter by the Russian Ministry of Education (MoE).

The major RBNet centres are: Moscow (including some centers in the so- called scientific towns in the Moscow Region – Dubna, Chernogolovka, Puschino and Obninsk), St.-Petersburg, Rostov-on-Don, Samara, Nizhny Novgorod, Ekaterinburg, Novosibirsk, Irkutsk and Khabarovsk. These centres are connected by 45 Mbps links with a star architecture (the center being in Moscow). In total RBNet has about 50 centres in all regions. In 2002-2003 the next 10 centers will be connected. The infrastructure is based on fiberoptic links from different operators (today mainly TRANSTELECOM; before the main provider was ROSTELECOM). Note that about 70% of the total traffic in RBNet relates to data exchange

6 internal to the network (to be compared with only 10% in 1997). The budget of RBNet is provided by MoIST, basically from the State Programme CNCN. Note that about 60% of the available RBNet capacity serves the universities.

RUNNet has about 20 centers at major sites of Russia, basically at regional universities and some scientific laboratories. This network is based on satellite links (from hundreds of kilobits/sec to Megabits/sec).2 This network is financed basically by MoE, with some small portion from the State Programme CNCN.

One should notice that there is an important joint project of RBNet and RUNNet, aimed at the creation of a high performance link between Moscow and St.-Petersburg. Designed for 622 Mbps, this link’s bandwidth is currently 155 Mbps (ATM).

6. International Links

In summer 2001, the capacity of the RBNet international link (with financial support from MoIST) was only 16 Mbps. There was also a project MIRnet (finished in 2001) that was supported by MoIST and NSF, with a 6 Mbps link to STAR-TAP (managed by Moscow State University and the University of Tennessee), that peered with ESnet in June 2000 to provide to the US Labs. There was also a 16-32 Mbps link between RUNNet and NORDUNet.

Also in summer 2001, a new project named FASTNet (see http://www.friends- partners.ru/friends/fastnet/home.html) started, with the goal of establishing a 155 Mbps international link. This project is funded jointly by MoIST and NSF. There was a tender, and the winning operator proved to be Teleglobe, who provided a 155 Mbps link to their Point-of Presence (PoP) in Frankfurt, and from there a 45-90 Mbps link to STAR-TAP. There was a plan to establish a connection to the new STARLIGHT access point in Chicago. The FASTNet link has technical support provided by RIPN, but the responsible institution is the Joint SuperComputer Center. FASTNet aims to provide high-performance and highly reliable connectivity between NSF laboratories and their counterparts in Russia. In the Spring of 2002, due to the bankruptcy of Teleglobe, a temporary agreement was signed with Telia, and the FASTNet link now has an intermediate PoP in Stockholm. The capacity of the FASTNet link is now 155 Mbps (STM1).

From summer 2002 another 155 Mbps link Moscow-Stockholm started (with Telia as the operator), for RBNet commodity Internet traffic.

At the end of 2002, a new tender was organized, to select the operator for the FASTNet link and the RBNet commodity Internet link. TRANSTELECOM won the tender. The likely start date is in March 2003.

From the Spring of 2002, discussions with GEANT risen to an intensive level, starting from the EC-Russia meeting in Brussels in May. As a result, an informal agreement was reached to establish a new Russia-GEANT link at 155 Mbps, and a formal Agreement for this link is currently being finalized. The financial conditions include the GEANT annual core fee, which will be covered by MoIST. This link will be provided by TRANSTELECOM or Telia, and managed by RBNet.

There are thus three STM1 (155 Mbps) links operating between Moscow and Stockholm. There is a plan to establish (later in 2003) an STM4 (622 Mbps) Moscow-Stockholm link that will connect to: one STM1 link to STARLIGHT (for FASTNet), one STM1 link to GEANT, and a third STM1 link for commodity Internet traffic. The current situation with the budget for the State Programme CNCN is that reliable financing of this plan (including the STM4 link to Stockholm and the three STM1 links) has now been secured. As for the fourth STM1 that could be

2 Some major Russian universities are served by RBNet, e.g. Moscow St. University and Novosibirsk St. University. 7 connected to the STM4, there are no plans at the moment, due to the remaining budget uncertainties.

For connectivity with the HEP centers in USA, the FASTNet link is used. Here we do not expect problems in 2003, and we expect this connectivity to adequately support our collaborative work with LHC partners in the USA, as well as other Grid-related applications.

For connectivity with the HEP centers in Europe, the new link with GEANT will be used. Probably the remaining fourth STM1 will serve this request (this would be the second link to GEANT in this case, but the corresponding discussions with GEANT will need to be carried out).

The bandwidth of the RUNNet-NORDUNet (Moscow-Petersburg-Helsinki) link is now 155 Mbps. In Helsinki the traffic is handed over to NORDUnet, for onward worldwide distribution. This link was used in 2002 by RBNet for international traffic after the Teleglobe bankruptcy. This link also was used in some Data Challenge sessions with CERN (managed by the cooperative network RUHEP). For this purpose a “VPN” (virtual circuit) was organized within the RUNNet-NORDUNet link by RUHEP and DESY. From 2003 this channel is planned to go in the link to GEANT.

The operational scheme for the FASTNet link, the commodity Internet link of RBNet, and the RUNNet-NORDUNet link, includes subdividing the capacity between different institutions and cooperative providers, including RUHEP, Moscow State University, JINR, the network of the Russian Academy of Science, and others. However, for Grid-related purposes, in particular for connectivity with European partners in the LHC Computing Grid project, there is a plan to organize virtual channels within the STM1 link to GEANT.

Further plans, for 2004-2006, depend strongly on the budget that will be available for those years.

7. Cooperative Networks

Over the last few years, a network of the Russian Academy of Science (RAS) has been developing swiftly. Now this cooperative network has its own fiberoptic cable infrastructure in Moscow. For example, the Institute of Theoretical and Experimental Physics (ITEP; an important HEP center in Moscow) now has 100 Mbps connectivity to the M9-IX through this network, with good prospects to increase this bandwidth up to 1 Gbps. In many regional cases, the RAS network uses the RBNet infrastructure.

An important example of the cooperative networks in Russia is RUHEP (http://www.ruhep.ru/ruhep/history.hep/histor_e.htm), which serves as an Internet provider for the high energy and nuclear physics institutes in Russia. In the past RUHEP organized, in close collaboration with DESY, a specialized network based on microwave links and satellite channels, both for internal and external connectivity. This project was financially supported by the German Government, INTAS and NATO. Now this cooperative network operates through a set of virtual channels set up within general-purpose links (basically through RBNet; also through RUNNet in some cases), or through use of a set of point-to- point links (a recent example is an IP tunnel between JINR and RUHEP).

8 8. Digital Divide Among HEP institutes High Energy Physics science and education in Russia is mainly centred in the regions of Moscow, St.-Petersburg and Novosibirsk. The main centres are:

1. Moscow: a. Institute for Theoretical and Experimental Physics (ITEP, http://www.itep.ru); b. Russian Research Center “Kurchatov Institute” (KIAE, http://www.kiae.ru); c. Skobeltsyn Institute of Nuclear Physics of Moscow State University (SINP MSU, http://www.sinp.msu.ru/eng/sinp.php3) ; d. Lebedev Physical Institute of RAS (LPI RAS, http://www. lebedev.ru/index.html.en); e. Moscow State Engineering Physics Institute (MEPhI, http://intraserv. mephi.ru/eng/second.html);

2. Moscow region: a. Joint Institute for Nuclear Research (JINR, http://www.jinr.ru), situated in Dubna, about 120 km to the North of Moscow. JINR has the status of an International Center. The member states are Russia and the Commonwealth of Independent States (known as NIS), and as well some states from Central and Southeast Europe who participate as member states or as an observers (e.g. Germany, Poland, Czech and Slovak Republics, Bulgaria); b. Institute for High Energy Physics (IHEP, http://www.ihep.su), situated in Protvino, about 120 km to the South of Moscow; c. Institute for Nuclear Research of Russian Academy of Science (INR RAS, http://www.inr.troitsk.ru), situated in Troitsk, about 40 km to the South of Moscow;

3. St-Petersburg region: a. Petersburg Nuclear Physics Institute of RAS (PNPI, http://www.pnpi.spb.ru), situated in Gatchina, about 40 km to the South of St. Petersburg;

4. Novosibirsk region: a. Budker Institutue of Nuclear physics of Siberian Branch of RAS (BINP SB RAS, http://www.inp.nsk.su), situated in Academgorodok, about 20 km from Novosibirsk. One should take into account that achieving high-performance and highly reliable connectivity with CERN, and the Laboratories participating in the LHC project remains a challenge, placing major demands on both the external connectivity and the regional links. The network demands of HENP have exceeded the capacity of international links in past years. Assuming that the plans put forward in the framework of the State Programme CNCN are successfully implemented, 2003 will mark the first time that the HEP requirements for international connectivity will be properly met. Setting up virtual channels in the link to GEANT with fixed bandwidth will be necessary, particularly for the peak periods, for LHC Data Challenges as well as for effective use of Grid testbeds. Participation in LCG project A Russian regional center is planned to be a coherent component of the international distributed infrastructure known as the LHC Computing Grid. It is designed as a cluster with the functionality of a Tier2 centre. The total resources will reach 70% of a canonical Tier1 centre for each LHC Collaboration. The architecture is based on the coherent interaction among the institute-based computing centres, through the use of DataGrid technologies. It means, in particular, that the connectivity between basic HEP institutes in Russia should be at the same level, in the Gbps range by 2006, as the international links.

9 The annual volume of data, produced by Russian HEP institutes in the framework of the program of Data Challenges, is estimated to be 25-30 Terabytes (TB)in 2003 and 50-70 TB in 2004. This is under the assumption that the Russian participation will be at the level of 5%. Each year these data have to be transmitted to CERN and other LCG partners in Europe and the USA. A similar volume of data should be copied back. It will be also necessary to take into account that some part of the data will need updating a few times per year (after detecting errors in the modelling programs or after their modifications). Thus, a rough estimation is that one can expect the data exchange between CERN (and other regional centres) to be 50, 120 and 250 TB in the years 2002, 2003 and 2004, respectively. Then the bandwidth for the data exchange with CERN (assuming 50% loading on the network links) should be at the level of 70-100 Mbps in 2003, and higher than 150 Mbps in 2004. In 2001-2002 the Russian institutes (JINR, IHEP, ITEP, SINP MSU, LPI RAS) were already actively participating in the CMS Monte Carlo simulation runs for the LHC, as well as in ATLAS and LHCb Data Challenges. In addition, the Russian institutes are involved in development and testing of the EU DataGrid Testbeds. In these sessions the data exchange was organized by the HENP cooperative network RUHEP, and the internal fiberoptic infrastructure of RBNet was used (mainly). For international connectivity the FASTNet link, the RUNNet-NORDUNet link and RBNet commodity Internet link (i.e. all three existing links) were used. Russian HEP institutes also are participating in the new EGEE initiative (creating the a Europe-wide Grid infrastructure), that is applying for financial support from the EC Framework 6th Programme.

The Network Status of Russian HEP Institutes

The Moscow HEP institutions use fiberoptic cables of different backbone infrastructures: RBNet, RAS and SMB. Basically, the Moscow institutions have 100-155 Mbps connectivity now. In 2001, a 1 Gbps link (GigaEthernet) between SINP MSU and ITEP has been created by TCSS, with the purpose being to test Grid technologies. RBNet and RAS also have 1 Gbps (Gigabit Ethernet) links now, in Moscow.

SINP MSU LAN uses several providers: RUHEP (for traffic to DESY, CERN and some other European centers), and MSUNet (the Moscow State University telecommunication center provides traffic to the USA and commodity Internet connectivity through RBNet and FASTNet). KIAE is served by RBNet. ITEP, LPI RAS and MEPhI are served by RUHEP.

At the end of December 2001, JINR (Dubna) got 30 Mbps of ATM connectivity to Moscow, using the fiberoptic cable of the State Satellite Communication Company. The link was organized by JINR and RBNet. In the first half of 2003, this channel will be upgraded to 155 Mbps, with plans to upgrade it progressively to the 1 Gbps level over the next years.

IHEP (Protvino) now uses the RUHEP microwave 8 Mbps link Protvino-Moscow. There is a project being implemented for a new fiberoptic link between Protvino and Moscow (M9-IX). This project plans to install a new fiberoptic segment owned by IHEP between Protvino and Moscow, and to link it to the existing fiber cable of GAZTELECOM. The Agreement assumes the mutual use of these two segments by IHEP and GAZTELECOM, free of charge for both sides: by IHEP for scientific traffic, and by GAZTELECOM for its commercial purposes in the Protvino area. The link should start operation in the summer 2003, with 100 Mbps of bandwidth for IHEP.

INR RAS, together with other institutes situated in Troitsk (Moscow region), uses a 2 Mbps fiberoptic link to Moscow, and two 2 Mbps microwave (RUHEP) links between Troitsk and Moscow. RUHEP is the Internet provider for INR RAS in both cases. There is a project (RAS) to upgrade the fiberoptic link to 34 Mbps in 2003.

10 PNPI (Gatchina): there is a fiberoptic cable to St.-Petersburg. However, it is private and too expensive, so that at the moment they use only a 256 kbps channel. RAS and RBNet have a project to put in place a new fiberoptic linking Gatchina and St. Petersburg in 2003, with a bandwidth of 34 Mbps (and TRANSTELECOM as the operator).

BINP SB RAS (Novosibirsk) uses a 34 Mbps RBNet Novosibirsk-Moscow terrestrial link now (TRANSTELECOM is the operator), and the traffic is routed by RUHEP.

Connectivity to KEK (Japan) for Russian HENP institutes is provided by a 512 Kbps terrestrial link between Novosibirsk and Tsukuba.

Brief Analysis of the reponses from Russia to the ICFA SCIC Questionnaire (Refer to the Appendix)

In the Appendix one finds the responses to the ICFA SCIC Questionnaire by five Russian HEP institutions. They give the complete picture of the Digital Divide in Russia among the scientific centers.

The highest level of network connectivity is illustrated by the answers from ITEP. A similar network situation applies for all Moscow institutions. In these cases 100-155 Mbps connectivity is now the norm, with good prospects to establish 1 Gbps links (to M9-IX) in the near future. This development (overcoming the Last Mile problem) can be covered financially by the internal budgets of these institutions.

The second category of HEP institutions is illustrated by the answers of IHEP (Protvino) and PNPI (Gatchina), with similar results for JINR (Dubna) and INR RAS (Troitsk). Here these centers are placed not so far from Moscow and St.- Petersburg (centers of the RBNet and RUNNet infrastructures, connected by the high-performance Moscow-Petersburg backbone, with easy increase to its capacity for a relatively small price). Thus, the Last Mile problem for these HEP centers consists of establishing proper links over the 40-150 km distance to the backbone. This requires a relatively large budget, and financial help is needed from State organizations, or from regular grants of International Funds. If this Last Mile problem is solved, then the overall problem of international connectivity will be solved for these institutes (but this is general problem that applied to all centers in Russia).

BINP SB RAS (Novosibirsk) has a special problem to obtain a high-performance link over a long distance (thousands of kilometers), over some rough terrain, to Moscow. Here a large budget is necessary, which can be provided only by the State budget or a special contribution of International Funds.

Finally the answers of Novgorod University show the problems of many regional universities and small laboratories. Note that Novgorod University is going to participate in the analysis of ATLAS data. In such cases, the main problems faced by the institutions are related to their Last Mile connectivity, but these problems cannot be solved by use of their internal sources alone. It could of course be solved by support from regional budgets. However, a new problem appears here – in many cases there is no possibility to reach the RBNet or RUNNet infrastructure directly. Instead the must go to regional commercial network providers, who as a rule charge exorbitant prices because they hold a virtual monopoly in the region.

However, the main problem for Russian science relates to the international connectivity. We do not see a clear way to provide reliable links with the bandwidth appropriate to support our current activity, and our future plans. This is certainly the case for participation in the Grid projects. As was said, this is due to absence of a regular State budget for these purposes.

11 9. Maps, Tables, Schemas

Russia Country Map Three regions are indicated on the map, where HEP centers are located: Moscow, St-Petersburg and Novosibirsk

12 The scheme of RBNet channels

13 The TRANSTELECOM network

The scheme of RUNNet channels

14 Russia: HENP Exp’s and Research Centre

Accelerator/Collider and Research Centres Site (Centre) Acc./Coll. HEP Fac. Other Exp’s Participation in major HEP Int. Collab. U-10 (fix target, proton beam Non-Acc. HEP CERN: ALICE, ATLAS, CMS, LHCb, ITEP 10 GeV) Exp’s. (Neutrino AMS (Moscow) Phys., etc) DESY: H1, HERMES, HERA-B, http://www.itep.ru TESLA

FNAL: D0, CDF, E-781(Selex) KEK: BELLE DAFNE: KLOE Nuclotron (heavy ions coll. at Low Ener. Acc., BNL: PHENIX, STAR JINR 6 GeV/n) Nuclear Reactor, CERN: ALICE, ATLAS, CMS, NA48, (Dubna, Moscow Synchrotron Rad.F., COMPASS, CLIC, DIRAC Region) Non-Acc. HEP DESY: H1, HERA-B, HERMES, http://www.jinr.ru Exp’s: Neutrino TESLA Phys., Medical FNAL: D0, CDF Exp’s, Heavy-ion KEK: E391a Physics U-70 (fix target, proton beam Medical Exp’s BNL: PHENIX, STAR IHEP 70 GeV) CERN: ALICE, ATLAS, CMS, LHCb (Protvino, Moscow DESY: ZEUS, HERA-B, TESLA Region) FNAL: D0, E-781(Selex) http://www.ihep.su VEPP-2M (linear collider at Non-Acc. HEP CERN: ATLAS, LHC-acc, CLIC BINP 1.4 GeV) Exp’s. (Neutrino FNAL: Tevatron-acc (Novosibirsk) VEPP-4 (linear collider up to Phys., etc), DESY: TESLA http://www.inp.nsk.su 6 GeV) Synchrotron Rad. F. KEK: BELLE SLAC: BaBar

15 Russia: HENP Exp’s and Research Centre (cont.)

Reseach Centres (Universities and Institutes) Site (Centre) HEP Acc./Coll. Other Exp’s Participation in major HEP Int. Collab. Low Energy Acc., CERN: ATLAS, CMS, AMS, CLIC SINP MSU Non-Acc. HEP Exp. DESY: ZEUS, TESLA (Moscow, University) (EAS-1000) http://www.sinp.msu.ru FNAL: D0, E-781(Selex) Low Energy Acc., BNL: PHENIX KIAE Nuclear Reactors, CERN: ALICE, AMS (Moscow, Res. Centre) Synchrotron Rad. F. http://www.kiae.ru Low Energy Acc., BNL: STAR MEPhI Nuclear Reactor CERN: ATLAS (Moscow, University) DESY: ZEUS, HERA-B, TESLA http://www.mephi.ru Low Energy Acc., CERN: ALICE, CMS, LHCb INR RAS Non-Acc. HEP Exp’s KEK: E-246 (Troitsk, (Neutrino Phys.) TRIUMF: E-497 Moscow region, Research Centre) http://www.inr.ac.ru Mid/Low Energy Acc., BNL: PHENIX PNPI Nulcear Reactor CERN: ALICE, ATLAS, CMS, LHCb (Gatchina, DESY: HERMES St-Petersburg region, FNAL: D0, E-781(Selex) Research Centre) http://www.pnpi.spb.ru

16 Moscow city map. Location of HEP centers are indicated, as well location of M9-Internet- Exchange Point (M9-IX)

17 Moscow Region Map. Location of HEP centers: JINR (Dubna), INR RAS (Troitsk) and IHEP (Protvino)

18 St.-Petersburg Region Map. Location of PNPI (Gatchina) is indicated

19 Novosibirsk Region Map. Location of BINP (Akademgorodok) is indicated

20 10. APPENDIX. ICFA SCIC Questionnaire – Responses by Russian Sites.

Table I – Respondents Name Institution e-mail Notes Serge Belov BINP SB RAS (Novosibirsk) [email protected] HEP Physicist Vadim Petukhov IHEP (Protvino) [email protected] Head of Math.and Computing Division Victor Kolosov ITEP (Moscow) [email protected] HEP Physicist Victor Abramovsky Novgorod St. University [email protected] HEP Physicist Yuri Ryabov PNPI RAS (Gatchina) [email protected] Chief of Information -Network and [email protected] computing connectivity responsibility

Table II - Connection Status Institution Bottleneck Notes Internal Regional / National International BINP SB No All Siberian networks is at The link BINP-KEK The link to KEK is Paid RAS, the 10-12 Mbps level, 0.5 Mbps is by KEK, SLAC and Novosibirsk including access to European currently the most BINP. and US networks. probable bottleneck. Upgrade cost prohibited. ITEP, No bottlenecks, No bottlenecks, No bottlenecks today, Moscow LAN 100-1000 100-1000 Mbps “Last Mile”, 155 Mbps to US & Mbps Regional 2-1000 Mbps. Europe IHEP, Local: 100 Mbps, Poor connection Protvino- Low throughput for Fast Ethernet Protvino LAN to WAN: Moscow. International 6 Mbps by a connections for the CISCO7500. scientific and federal Possibility to 1Gbps organizations Novgorod Here is the main And outside of the LAN - WAN uses State bottleneck University- Max of 30 Kbps firewall with max of 2 University Mbps. PNPI RAS, 10-100 Mbps 256 Kbps Gatchina Ethernet

Table III - Connection Details Institution Providers Firewall Regional / National (2) International (3) IHEP, Fast Ethernet, Ministry of Industry, Science and No Protvino RUHEP link owned by IHEP Technologies Novgorod State All is paid by the University Yes University BINP SB RAS, National network, which is Connection to RBNet is 34Mbps. Yes Novosibirsk provided by 34M trunk The link Novosibirsk-Moscow is operated by shared by numerous organizations TRANSTELECOM. of Siberian Branch of Russian CANET (city network), Academy of Science and SANET2 (national network). universities. PNPI RAS, External channel is covered by Gatchina institution’s budget and RAS

Tabel IV - Other Networking Needs Institution Computing / networking needs related to HEP Other BINP SB RAS, Upgrade everything. Scarce resources, equal situation for other SB RAS with the funds Novosibirsk communities, it is very difficult situation for upgrades of get a reserved by the Ministry of dedicated network. Industry, Science and Technologies ITEP, Moscow Routing equipment for LAN, and to increase the capacity of

21 external connections. IHEP, Protvino Local area network: our plan is to reach 1 Gbps within the next Connection to WAN: 600 3 years Link to Moscow will grow to 30 Mbps by the next year Mbps to be supported by the and up to 600 Mbps by 2007-2008 ( optical link) budget for Institute Novgorod State First half of 2003 increase existing external channel throughput University to no less than 150 Mbps. Reconstruct local area network to Gigabit Ethernet.

Table V - Most Relevant Networking-Related Problems Institution Most relevant networking related problems BINP SB RAS, 0.5M dedicated channel BINP-KEK, 100M connection to regional network. Novosibirsk 34M shared connection to RBNet (up to 10% really available for BINP) 10M shared connection to Global Internet (up to 10% really available for BINP) Future needs: 2M dedicated channel BINP-KEK + (2-3) Gbps connection to regional network + (4-5)*155 M shared connection to RBNet or 155M dedicated connection to RBNet + 155M dedicated connection to Global Internet. IHEP, Protvino International connections: low bandwidth caused by high prices Novgorod State We have only financial problems University PNPI RAS, Next requirements for international networks Gatchina

Table VI - Presented Ideas for Prospective Solutions Institution Presented ideas toward a solution for the detected problems BINP SB RAS, No specific technical ideas/inventions - all technologies are well proven and the only thing Novosibirsk necessary in order to implement them is additional funding. IHEP, Protvino In many cases we need access only to the some particular Scientific Centers (CERN, FNAL, DESY, …) but not to all Internet resources. Novgorod State We have just began, thus the only problem that we met is lack of facilities, due to the lack of University funds allowed for this program.

A-Mainframe, Clusters, LAN, Networked CPUS Mainframe Clusters Networked CPUs LAN Technology Instituti on BINP SB RAS, No (4-5) clusters working for Intracluster Usually 100M connection Novosibirsk different projects, each of 10- interconnections are via Ethernet switch. 15 middle-range CPU based on switched (P3/500MHz, 1GB Memory, 100M/1G Ethernet. 20GB/CPU storage). ITEP, Moscow No 50 CPUs, 2 TB Disk Storage 100-1000 Mbps Ethernet local connections: 100 Mbps Ethernet, and 1 GBit Ethernet backbone optical link IHEP, Protvino AlphaServers: (1) 10 CPUs Pentium III 833 Fast Ethernet; 10 Mbps Ethernet; (1) 8200 MHz; FDDI; 100 Mbps Ethernet; (6CPUs) 5/300 (2) 30 CPUs Pentium III 930 Ethernet FDDI MHz; 1GB MHz; RAM; (2) (3) 2 TB discs; 8200 (4) Fast Ethernet. (6CPUs)5/440 MHz;1GB RAM; (3) 200 (4CPUs) 5/625 MHz; 1GB RAM.

22 Novgorod State No No One CPU connected to We have only one University the LAN, it is dedicated computer connected to the to HEP University’s switch using Fast Ethernet (100 Mbps). PNPI RAS, No Our LAN consists of more The segments of LAN Technologies using cooper Gatchina then 500 PC (~200 PC belongs to different twisted pairs (5th cat) and dedicated to HEP). Cluster departments built on the communication equipment dedicated to HEP: 6 dual PC 10Mbps Ethernet and HUB and Switch (Cisco, (Intel p-III-700 Mhz, RAM- Fast Ethernet 3COM, Intel and others). 256 Mb, HDD 30 GB). The segments are combined into Institution’s LAN using optical Fiber (the Length ~ 5 km).

B-Firewall, WAN, Hops, Financial Support Institution Firewall WAN connection hops Who pays for the connection ? BINP SB RAS, No - 15 The Institute Novosibirsk ITEP, Moscow No info. No Info. Regional 5-10 No info. Internet 10-20 IHEP, Protvino No Info. No Info 10 IHEP Novgorod State Yes. No info. 30 The University University PNPI RAS, No Info. Our LAN is connected to the commercial Gatchina WAN using optical fiber. HEP community of No Info. Russian Academy of our institute hasn’t any special lines to the Science WAN. The bandwidth PNPI-Gatchina ~ 256 Kbps, Gatchina - Saint-Petersburg - 2048 Kbps (current status). The bandwidth PNPI - Gatchina restricted by financial reason.