Comparative Trends in Research Performance of the Russian Universities

Scientometrics (2018) 116:2019–2052

https://doi.org/10.1007/s11192-018-2807-6(0123456789().,-volV)(0123456789().,-volV)

Comparative trends in research performance of the Russian universities

Anna A. Avanesova1 • Tatyana A. Shamliyan2

Received: 5 March 2018 / Published online: 14 June 2018 Ó Akade´miai Kiado´, Budapest, Hungary 2018

Abstract We analyzed comparative trends in research performance of the Russian institutions based on the quantitative and qualitative data available in the international SciVal, Scopus, web of science and the national Russian databases from 2012 to December 2017. Russian Federation represented 2.0% of the world population and 2% of the researchers while accounting to 2.2% of the world publications (14th place in the world scholarly output) with overall field-weighted citation impact of 0.75. Scholarly output of the Russian authors increased by 79%, field-weighted citation impact by 12% and outputs in top citation percentiles by 21% but without a statistically significant positive association between higher investment in research and development and the increase the national GDP. Scholarly output for the Russian publications in mathematics, physics and astronomy are among 5 top countries. However, field-weighted mass media impact, the number of citations per publication, citations per author and per publication, metrics of international collaboration and the academic–corporation collaboration and economic impact of the Russian research remain low. Routine analysis of the research performance and economic impact of R&D expenditure should be reflected in transparent distribution of state research funding. Legal aspects of the international research must be developed to ensure a complete integration of the Russian science into international research activities.

Keywords Academic institutions Á Productivity Á Internationalization Á Russian Federation Á Artiﬁcial intelligence

& Tatyana A. Shamliyan [email protected] Anna A. Avanesova [email protected]

1 North-Caucasian Federal University, Pushkina street, 1, Stavropol, Russian Federation 355009 2 Quality Assurance, Evidence-Based Medicine Center, Elsevier, 1600 JFK Blvd, Philadelphia, PA 19103, USA 123 2020 Scientometrics (2018) 116:2019–2052

Introduction

Science-driven innovation is essential for the national economic success (O’meara 2015; Asheim et al. 2016). State investment in higher education and facilitation of foreign direct investments into multidisciplinary international research activities improve research performance and increase innovation capabilities of all contributors across the borders (Chen et al. 2017; Salter et al. 2016;Li2011; Kulmala et al. 2015). Russian government implemented several programs supporting leading universities in science-based innovation (Schiermeier 2010; Decree of the Government of the Russian Federation 2013; Decree of the President of the Russian Federation 2012; The Directive of the Government of the Russian Federation 2012, 2013; Federal Assembly Of The Russian Federation 2017). The Russian Academic Excellence Project and Project 5-100 aimed at improving ‘‘the competitive position of the leading Russian universities in the global research and education market’’ (Turko et al. 2016; The Ministry of Education and Science of the Russian Federation). These initiatives improved some parameters of research performance of the Russian universities (Markusova et al. 2013; Pislyakov and Shukshina 2014; Lyagushkina and Bogatov 2017). However, the place of Russia in the modern global scientific community remains unexplored. Overall productivity, scientific, economic and societal impact of the Russian research institutions is unclear. Comparative trends in specific research performance indicators and the most current ranking of Russian contribution in various scientific subjects have not been analyzed.

Methods

We conducted a comprehensive analysis of international SciVal, Scopus, web of science and the Russian electronic library databases to identify publications of the authors affiliated with Russian Federation, the USA, China and other countries (Li et al. 2017; Reznik-Zellen 2016). We analyzed research performance data from 2012 to December 2017 to address a lag between increased Russian investment in the national research performance (Schier- meier 2010). We analyzed country’ investment in research and development using the data from the Organization for Economic Co-operation and Development (https://data.oecd.org). We analyzed the value of awarded grants based on the data available in SciVal from the following funding entities: Welcome Trust, the National Science Foundation, National Institutes of Health, Arts and Humanities Research Council, Biotechnology and Biological Sciences Research Council, Economic and Social Research Council, Engineering and Physical Sciences Research Council, Medical Research Council, Natural Environment Research Council, Science and Technology Facilities Council, Australian Research Council, National Health and Medical Research Council (Colledge and Verlinde 2014). We analyzed Patent Article Citations data available in SciVal from European Patent Office, Intellectual Property Office, Japan Patent Office, United States Patent and Trade- mark Office and the World Intellectual Property Organization (Colledge and Verlinde 2014). We analyzed media exposure available in SciVal from the LexisNexis Metabase portal (http://www.lexisnexis.co.uk/en-uk/terms.page). We analyzed the dynamic of share of Russian publications in the global number of publications in Scopus and the position of Russia in the global ranking of countries by number of publications in Scopus. We compared country population, spending on research 123 Scientometrics (2018) 116:2019–2052 2021 and development, research productivity and impact of Russia in comparison with other countries. We tested hypotheses of the association between spending on research and development and scientific productivity. We tested hypotheses of the statistical significance of annual trends in research performance indicators and the differences between indicators of the Russian and The US institutions and authors. We compared research performance between the Russian Fed- eration and the USA because the USA is a recognized leader in diverse research fields. We analyzed the scientific collaboration of Russia with other countries. We examined how the intensity of scientific collaboration varies by disciplines. We explored what countries and what foreign research organizations and universities are key scientific partners of Russian authors. We ranked indicators of research performance in different research areas across countries of author’ affiliation. We also ranked indicators of research performance among the Russian institutions. We analyzed correlations between various performance and economic impact indicators. We extracted the data on publication activity of the Institutes of Russian Academy of Sciences, other research universities, and other research organization of enterprise sector (Russian Academy of Science 2013). To address differences in research productivity, collaboration, scientific and economic impact, we analyzed the Russian Science Citation Index (Mongeon and Paul-Hus 2016; Gorin et al. 2016; Bolotov et al. 2014) and the following indicators of all publications indexed in Scopus (Table 1). For statistical analyses we used STATA (StataCorp http://www.stata.com). Significance was analyzed at 95% confidence level.

Results

Russian Federation represented 2.0% of the world population and 2% of the researchers while accounting to 2.2% of the world publications with overall Field-Weighted Citation Impact of 0.75 (Figs. 1, 2). In comparison, the US represented 1.5% of the world population but 18% of the researchers accounting to 22.8% of the world population with overall Field-Weighted Citation Impact of 1.45. Switzerland and Netherlands represent 0.1 and 0.2% of the world population with the highest Field-Weighted Citation Impact of 1.83 and 1.79 respectively (Figs. 1, 2). We analyzed the association between countries investment into research and development and research performance. We found that country’s gross domestic spending on research and developments is positively associated with the number of the authors and publications but not with citations per publications and Field-Weighted Citation Impact indices. However, higher percentage of spending on research and development was positively associated with higher Field-Weighted Citation Impact indices (p value 0.032). In 2012–December 2017, scholarly output of the Russian authors represented only 9.4% of the US of scholarly output (Table 2). However, during this period of time the Russian scholarly output increased by 79% while the US scholarly output decreased by 1.4% (Table 2). Increase in the Russian scholarly output was incrementally higher than the Russian investment in research and development. For instance, in 2015, Russia invested in research and development 1.1% of GDP with the total GDP of $24,710/capita. At the same year, the US invested in research and development 2.8% of GDP with the total GDP of $56,420/capita. Overall during 2000–2015, there was a statistically signiﬁcant positive 123 2022 Scientometrics (2018) 116:2019–2052

Table 1 Indicators of research productivity, collaboration, scientiﬁc and economic impact Indicator Measured entity Deﬁnition characteristic

Scholarly output Productivity The number of indexed in Scopus publications Field-weighted citation Scientific impact The number of citations received by an impacta entity’s publications compared with the average number of citations received by all other similar publications in the data universe Indicator Measured entity Definition characteristic Citations Scientific impact The number of citations received by an entity’s publications Citations per publication Scientific impact The average citation impact of the publications as the number of an average received citations Outputs in top citation Scientific impact The number of publications in the top 1, 5, percentiles (top 10%) 10 or 25% of the most-cited publications Outputs in top views Scientific impact The number of publications are in the top percentiles (top 10%) 1, 5, 10 or 25% of the most-viewed publications Publications in top journal Scientific impact The number of publications are in the top percentiles (top 10% by 1, 5, 10 or 25% of the most-cited journals CiteScore percentile) indexed by Scopus Topic prominence Scientific impact Is an indicator of momentum/movement or visibility of a particular topic. Topic prominence is comprised of three metrics —recent citation counts, recent views counts, and journal impact (CiteScore). These three metrics are computed and then normalized using log transforms and standard deviations. The results are then combined as a weighted average Academic–corporate The degree of collaboration Proportion of co-authored publications collaboration (%) between academic and across the academic and corporate, or corporate affiliations industrial sectors International collaboration The degree of collaboration Proportion of internationally co-authored (%) between international publications coauthors Patent-citations count (patent Economic impact The total count of patent citations received office: all patent offices) by the entity Patent-citations per scholarly Economic impact The average patent-citations received per output (patent office: all 1000 scholarly outputs published by the patent offices) entity Citing-patents count (patent Economic impact The count of patents citing the scholarly office: all patent offices) output published by the entity Indicator Measured entity Definition characteristic Patent-cited scholarly output Economic impact The count of scholarly output published by (patent office: all patent the entity (e.g. a university) that have offices) been cited in patents

123 Scientometrics (2018) 116:2019–2052 2023

Table 1 continued

Indicator Measured entity Deﬁnition characteristic

Mass media (print) Social impact Total count of mass media mentions for institutions and countries Field-weighted mass media Social impact The number of media mentions received by (print) an entity’s publications compared with the average number of media mentions received by all other similar publications in the data universe Media exposure Social impact The ratio of media mentions weighted by type of publication, demographics and audience reach Topic prominence Scientiﬁc impact Is an indicator of momentum/movement or visibility of a particular topic. Topic Prominence is comprised of three metrics –recent citation counts, recent views counts, and journal impact (CiteScore). These three metrics are computed and then normalized using log transforms and standard deviations. The results are then combined as a weighted average h-indices Productivity and scientiﬁc A balance between the productivity impact (Scholarly Output) and citation impact (Citation Count) of an entity’s publications aA Field-Weighted Citation Impact of 1.00 indicates that the entity’s publications have been cited exactly as would be expected based on the global average for similar publications; the Field-Weighted Citation Impact of ‘‘World’’, or the entire Scopus database, is 1.00

association between higher investment in research and development and the increase the national GDP (p value 0.004) in the USA. In contrast, the Russian investment into research and development as well as the national GDP did not demonstrate signiﬁcant increase during these years. Despite no increase in research and development expenditure, Field-Weighted Citation Impact of the publications authored by Russian scientist increased by 12% in 2012–2017 although remaining lower than the US FWCI (0.73 versus 1.45 respectively) (Table 2). Field-Weighted View Impact of the Russian publication was slightly higher when compared with the US (1.16 versus 1.08 respectively, Fig. 3). Field-Weighted Mass Media impact remains substantially low (0.05) when compared with the US (2.01) (Fig. 3) suggesting poor media coverage of the Russian scientiﬁc publications. Despite increased number of authors, the total number of citations and the number of citations per publication decreased since 2012 in Russia and in the US (Table 2). The total number of the Russian authors and citations are much smaller when compared with the US (Table 2). This difference in authors and citations are unexpectedly higher than the total number of researches per 1000 employed in Russia (6.2) compared with the US (9.1). It may suggest that many Russian researchers either do not actively publish their results or are not represented in the international databases.

123 2024 Scientometrics (2018) 116:2019–2052

United States 4.5 22.8 China 16.7 19.3 United Kingdom 0.9 6.8 Germany 1.1 5.9 India 4.6 17.8 Japan 1.8 4.6 France 0.9 4.1 Italy 0.8 3.8 Canada 0.5 3.6 Australia 0.3 3.2 Spain 0.6 3.1 South Korea 0.7 2.8 Brazil 2.42.7 Russian Federation 2.02.2 Netherlands 0.2 2.0 Iran 1.11.7 Switzerland 0.1 1.5 Poland 0.5 1.5 Turkey 1.11.5 Taiwan 0.3 1.4 0.0 5.0 10.0 15.0 20.0 25.0

% World Publicaons % World Populaon

Fig. 1 Percentage of world population and publication among 20 countries with the largest scholarly output (OESD and Scopus 2013–2014)

Outputs in top citation percentiles demonstrated a positive trend for the publications by the Russian authors (increase by 21%) although it remained lower when compared with the publications of the US authors (Table 2, Fig. 4). Publications in top journal percentiles slightly increased for the US authors in contrast with 22% decrease for the Russian authors (Table 2). Citations per authors and citations per publication are higher in the US than in Russia (Fig. 5) with a negative trend observed in the Russian and in the US publications. Similar decrease was observed in metrics of international collaboration (Fig. 6) and the academic–corporation (Fig. 7) collaboration of the Russian authors (Table 2). In contrast, a growing number of the US researchers authored publications in collaboration with researchers from other countries (Table 2). We analyzed the association between international collaboration and performance indicators across all countries. International collaboration was positively associated with the publications in top journals (correlation coefﬁcient [ 0.7). Increase overtime in international collaboration was not associated with research performance indicators across all countries. The intensity of scientiﬁc collaboration differed among 205 countries that had publications coauthored with the Russian authors. For instance, 5 countries represented 86% of all international Russian publications including United States (25%), Germany (24%), France (14%), United Kingdom (13%) and Italy (10%). However, other regions 123 Scientometrics (2018) 116:2019–2052 2025

United States Thailand China Romania 2 United Kingdom New Zealand 1.56 Germany 1.81.45 Egypt 1.49 India 1.6 1.42 Hong Kong Japan 1.66 1.4 0.93 0.89 Saudi Arabia 0.86 1.2 France 1.45 0.93 1.34 Greece 1 0.77 0.96 Italy 1.5 1.35 0.8 South Africa Canada 0.6 1.5 1.26 0.4 Finland1.66 1.56 Australia 0.2 Singapore 1.75 0 1.29 Spain 1.02 Israel 1.47 South Korea 0.87 0.75 Norway 0.88 Brazil 1.66 1.09 Mexico 0.95 Russian Federaon 1.27 0.8 Czech Republic 0.91 1.02 1.79 Netherlands 0.99 Portugal Iran 1.57 1.83 Austria Switzerland Denmark Poland Malaysia Turkey Belgium 1.67 Taiwan 1.83 Sweden 1.7

Fig. 2 Field-weighted citation impact among 40 countries with the largest scholarly output (Scopus 2012–2018) collaborated with the Russian Federation as well (Table 3). The intensity (% growth) of collaboration was the largest with the Asian and African countries. The intensity of scientific collaboration differed among research areas. The number of co-authored publications was the highest in the areas of physics and astronomy (40,158 publications), materials science (18,479 publications), engineering (15,648), chemistry (12,834 publications), earth and planetary sciences (10,264 publications), biochemistry, genetics and molecular biology (9729 publications), and mathematics (9580 publications). The intensity of scientific collaboration differed among specific institutions. The Rus- sian Academy of Science and affiliated institutions, Moscow State University, Joint Institute for Nuclear Research, Alikhanov Institute for Theoretical and Experimental Physics were among the most active in international collaboration. Massachusetts Institute of Technology and Harvard University in the USA, University of Heidelberg, Karlsruhe Institute of Technology, Electron Synchrotron in Germany, University of Oxford, Imperial College London, University of Manchester in the United Kingdom, Universite Paris Saclay, The French National Center for Scientific Research in France were among the most active in the collaboration with the Russian researchers. Economic impact of the Russian research remains much smaller when compared with the US without positive trends for both countries (Table 2). Many parameters of economic funding and impact were not available for the Russian Federation. For instance, the number

123 2026 Scientometrics (2018) 116:2019–2052

Table 2 Comparison of research performance between Russian Federation and United States (Scopus, 2012–2017) Parameter Country Overall % % change in 2016 Russian/ versus 2012 USA

Scholarly output Russia 355,452 178.8 USA 3,800,417 9.4 98.6 Scholarly output (growth %) Russia 78.8% USA - 1.4% Field-weighted citation impact Russia 0.73 111.8 USA 1.45 50.3 98.0 Citations Russia 1,097,261 33.0 USA 29,869,806 3.7 18.2 Citations per publication Russia 3.1 18.8 USA 7.9 39.2 18.4 Authors Russia 283,806 179.6 USA 2,851,649 10.0 115.3 Authors (growth %) Russia 79.6% USA 15.3% Outputs in top citation percentiles (top 10%) Russia 7.0% 121.2 USA 18.3% 38.3 100.6 Outputs in top views percentiles (top 10%) Russia 10.4% 183.3 USA 10.8% 96.3 93.9 Publications in top journal percentiles (top 10% Russia 11.3% 78.0 by CiteScore percentile) USA 37.4% 30.2 101.9 Academic–corporate collaboration (%) Russia 1.2% 78.6 USA 3.2% 37.5 97.0 International collaboration (%) Russia 26.2% 82.5 USA 31.4% 83.4 123.4 Patent-citations count (patent office: all patent Russia 1405 0.0 offices) USA 85,143 1.7 0.1 Patent-citations per scholarly output (patent Russia 4 0.0 office: all patent offices) USA 22.4 17.9 0.1 Citing-patents count (patent office: all patent Russia 1263 4.3 offices) USA 44,942 2.8 4.1 Patent-cited scholarly output (patent office: all Russia 707 0 patent offices) USA 36,777 1.9 0.0234 Mass media (print) Russia 6718 29.0 USA 2,019,401 0.3 33.5 Field-weighted mass media (print) Russia 0.05 0.0 USA 2.01 2.5 90.9 Media exposure internationally recognized Russia 1343 52.3 (print) USA 201,741 0.7 61.5 Media exposure locally recognized (print) Russia 359.4 26.5 USA 281,812.4 0.1 31.6

123 Scientometrics (2018) 116:2019–2052 2027

Table 2 continued

Parameter Country Overall % % change in 2016 Russian/ versus 2012 USA

Media exposure nationally recognized (print) Russia 563.4 45.6 USA 62,197.2 0.9 29.7 Media exposure regionally recognized (print) Russia 849.5 12.5 USA 100,142 0.8 26.8

2.5

1.5

0.5

0 2012 2013 2014 2015 2016 2017

FWCI-Russia FWCI-USA FWMM-Russia FWMM-USA FWVI-Russia FWVI-USA

Fig. 3 Trends in field-weighted citation impact factor in the russian federation and the in the US. FWCI field-weighted citation impact; FWMM field-weighted mass media (print); FWVI field-weighted view impact; FWVI field-weighted view impact and monetary value of awards were not reported for the Russian publications in SciVal. Total country’ investment and added value of information and communication technology were not reported for the Russian Federation in oecd.org. Dissemination of the Russian research in media remains much smaller when compared with the US without positive trends for both countries (Table 2). We compared research performance of the Russian Academy of Science, research institutions not affiliated with the RAS and research organizations of enterprise sector. The authors affiliated with the Russian Academy of Science constituted 18% of all Russian authors but contributed 27% of publications and 32% citations (Fig. 8). The percentage of the authors affiliated with the Russian Academy of Science decreased over the last years while the total number of citations did not (Fig. 8). Research organizations of enterprise sector constitute a very small proportion of the Russian research organizations although they demonstrated a large author growth of 400–800% (e.g. Atom Energoproekt Ltd or Advanced Energy Technologies Ltd). 123 2028 Scientometrics (2018) 116:2019–2052

30.00%

25.00% R² = 0.7605

20.00%

15.00%

R² = 0.8477 10.00%

5.00%

0.00% 2012 2013 2014 2015 2016 2017 Russia USA Poly. (Russia) Poly. (USA)

Fig. 4 Trend in outputs in top citation percentiles (top 10%) if the Russian and US publications

7.9

3.1

4 Citaons per author Citaons per Publicaon Citaons per

024681012

USA Russia

Fig. 5 Citations per author and per publication in the Russian and US publications (Scopus, 2012–2017)

We compared research performance among 223 Russian and 1252 US institutions and 500 top authors from both countries. Publications and author growth were larger in the Russian Federation while citations per publication and Field-Weighted Citation Impact 123 Scientometrics (2018) 116:2019–2052 2029

40.00%

38.00% R² = 0.9948 36.00%

34.00%

32.00%

30.00% R² = 0.6429 28.00%

26.00%

24.00%

22.00%

20.00% 2012 2013 2014 2015 2016 2017

Russia USA Poly. (Russia) Linear (USA)

Fig. 6 International collaboration (%) in the Russian and US publications

3.50%

3.00%

2.50% R² = 0.665

2.00%

1.50%

1.00% R² = 0.6857 0.50%

0.00% 2012 2013 2014 2015 2016 2017

Russia USA Linear (Russia) Linear (USA)

Fig. 7 Academic–corporate collaboration (%) the Russian and US Publications were larger in the US (Table 4). An average hi indices were higher for the top 500 US authors (74.8) when compared with Russian authors (51.5). Scholarly output and Field-Weighted Citation Impact varied among different research areas. Field-Weighted Citation Impact from the Russian publications was higher than 1 (world data) for the publications in engineering, medicine, dentistry, veterinary, arts and humanities, economics, econometrics and ﬁnance, business, management and accounting and decision sciences (Table 5). Scholarly Output for the Russian publications in mathematics, physics and astronomy are among 5 top countries (Table 5). Scholarly Output for the Russian publications in earth and planetary sciences, materials science, engineering, chemistry and chemistry engineering as well as pharmacology, toxicology and pharmaceutics are among 10 top countries (Table 5).

123 2030 Scientometrics (2018) 116:2019–2052

Table 3 A distribution of co-authored with the Russian researches publications by the subject area and geographic continent of collaborators (Scopus, 2012–2017) Scopus subject area South North Middle Europe Asia Africa Totala %b America America East Paciﬁc

Physics and astronomy 3467 11,797 3978 30,379 11,184 2189 62,994 41.4 Materials science 241 3442 827 13,279 4213 217 22,219 19.1 Engineering 521 3140 963 10,833 3972 472 19,901 16.1 Chemistry 147 2446 674 9050 2766 171 15,254 13.2 Earth and planetary 570 3412 533 6971 3039 471 14,996 10.6 sciences Biochemistry, genetics and 321 3708 684 6344 2189 267 13,513 10.0 molecular biology Medicine 611 3361 909 5736 2040 434 13,091 9.9 Mathematics 325 2141 601 6411 1845 174 11,497 8.5 Agricultural and biological 277 2292 587 5042 2285 355 10,838 8.2 sciences Computer science 116 1508 321 4801 1403 89 8238 7.2 Environmental science 104 1163 196 2788 1161 149 5561 4.3 Chemical engineering 53 736 271 2739 985 51 4835 4.2 Social sciences 66 806 152 1901 607 58 3590 3.1 Energy 27 621 144 1909 743 66 3510 3.0 Immunology and 97 719 152 1358 524 115 2965 2.2 microbiology Multidisciplinary 64 627 120 1172 519 63 2565 1.7 Pharmacology, toxicology 48 550 100 990 386 28 2102 1.7 and pharmaceutics Neuroscience 57 553 93 951 230 23 1907 1.4 Arts and humanities 21 339 71 806 206 16 1459 1.2 Psychology 50 373 64 489 139 36 1151 0.9 Economics, econometrics 12 208 27 551 146 7 951 0.8 and ﬁnance Business, management and 22 175 33 514 139 13 896 0.8 accounting Decision sciences 16 153 54 499 139 18 879 0.8 Health professions 17 98 27 255 54 12 463 0.4 Nursing 14 56 27 119 53 10 279 0.2 Veterinary 10 44 5 74 30 5 168 0.1 Dentistry 6 4 20 6 2 38 0.0 Total number of 7274 44,474 11,617 115,981 41,003 5511 publications aTotal number of publications coauthors by Russian researchers in each subject area bPercentage of the publications of speciﬁc subject area in total number co-authored with the Russian researches publications for all regions

123 Scientometrics (2018) 116:2019–2052 2031

35.0 33.0 32.6 32.1 32 31.0 30.0 30.0 29.1 27.6 27 25.3 25.6 25.7 25.4 25.0 22.7

20.4 19.9 19.8 20.0 18

15.0

10.0

5.0

0.0 2013 2014 2015 2016 2017 Overall

Scholarly Output Citaons Authors

Fig. 8 Percentage of scholarly output, citations and authors from the Russian Academy of Science among all Russian publications

Table 4 Differences in research performance between Russian and US institutions and authors (Scopus, 2012–2017) Parameter Russia USA Cohen standardized mean difference

Mean SD Mean SD Mean Lower 95% Upper 95% CI CI

Analysis of institutions Publications 2283 7537 4803 11,427 - 0.2 - 0.4 - 0.1 Publications (growth %) 198.0 355.3 9.8 258.1 0.7 0.5 0.8 Authors (growth %) 197.8 433.6 17.2 231.1 0.7 0.5 0.8 Citations per publication 3.2 3.6 9.0 7.9 - 0.8 - 0.9 - 0.6 Field-weighted citation 0.8 0.6 1.6 0.8 - 1.0 - 1.2 - 0.9 impact Analysis of 500 top authors Citations per publication 25.6 14.6 32.8 3.9 - 0.7 - 0.8 - 0.5 Field-weighted citation 3.2 1.3 4.0 0.3 - 0.9 - 1.0 - 0.8 impact h-index 51.5 18.6 74.8 12.6 - 1.5 - 1.6 - 1.3

Publications, citations, citations per publication, author growth and FWCI varied among Russian institutions (Appendix Table 6). Leading Russian universities participating in the Program ‘‘5–top 100’’ did not demonstrate the best research performance when compared with other institutions, e.g. Institute for High Energy Physics, Saint Petersburg Nuclear Physics Institute, or Alikhanov Institute for Theoretical and Experimental Physics. The top 123 2032 Scientometrics (2018) 116:2019–2052

Table 5 Ranking of Russian research performance parameters by subjects of research (SCOPUS catego- rization, 2012–2017) Subject area Scholarly % of world scholarly Field-weighted citation outputb output impactb

Physics and astronomy 29,292 (5)a 5.5 0.94 (29) Mathematics 10,806 (4)a 4.9 0.69 (67) Earth and planetary sciences 5,453 (6)a 4.0 0.75 (62) Energy 1,425 (21) 1.2 0.75 (81) Materials science 20,667 (9)a 2.9 0.88 (59) Engineering 952 (10)a 2.1 1.73 (60) Chemistry 22,266 (9)a 2.8 0.56 (84) Biochemistry, genetics and 9,074 (16) 1.7 0.6 (94) molecular biology Chemical engineering 9,927 (10)a 2.4 0.43 (92) Computer science 7,601 (16) 1.7 0.95 (46) Environmental science 4,533 (14) 2.1 0.89 (75) Medicine 12,334 (23) 0.8 1.15 (87) Health professions 6 (65) 0.1 0.44 (76) Nursing 10 (84) 0.02 0.82 (69) Dentistry 46 (62) 0.1 1.12 (51) Immunology and microbiology 961 (20) 1.0 0.81 (90) Pharmacology, toxicology and 1,338 (9)a 1.9 0.48 (90) pharmaceutics Neuroscience 1,667 (17) 1.1 0.48 (94) Agricultural and biological sciences 4,234 (20) 1.0 0.62 (93) Veterinary 214 (58) 0.2 1.21 (47) Arts and humanities 2,002 (12) 1.3 1.58 (6) Multidisciplinary 4,794 (14) 1.8 0.85 (72) Economics, econometrics and 3,685 (10)a 2.9 1.46 (11) ﬁnance Business, management and 2,406 (13) 1.9 1.14 (44) accounting Decision sciences 198 (24) 0.9 1.27 (45) Psychology 1,017 (22) 0.8 0.34 (93) Total value for the Russian 355,452 2.2 0.75 Federation aRussian Federation is among 10 top countries bRank among other countries

performing research organization according to productivity indicators included The Rus- sian Academy of Science, Moscow State University and St. Petersburg State University (Appendix Table 6). However, the most productive universities did not demonstrate the highest scientiﬁc impact of their publications. Rostov State University of Economics had the highest FWCI and publication growth and Institute for High Energy Physics had the FWCI and the number of citations per publication (Appendix Table 6).

123 Scientometrics (2018) 116:2019–2052 2033

35000

R² = 0.9717 30000

25000

20000 R² = 0.9782

15000

10000

5000 R² = 0.9511

0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

Russia USA China Poly. (Russia) Poly. (USA) Poly. (China)

Fig. 9 Trends in publication regarding artiﬁcial intelligence afﬁliated with US, Chinese, and Russian institutions

We analyzed the Russian Index of Science citations (Appendix Table 7) and found no statistically significant association with Field-Weighted Citation Impact. Universities with the highest value of the Russian Index of Science citations did not have the highest corresponding values of Field-Weighted Citation Impact (R squared 0.03). We could not analyze the association between research performance and state or industry research funding because this information is not available for the Russian Fed- eration. Prioritization of research areas is based on presidential decrees and the general conception of multidisciplinary research and modern technologies (Meyer 2017; Russian Academy of Science 2013). We, therefor, analyzed research performance in the area of artificial intelligence as an example of emerging high impact research subject supported by the Russian President (Meyer 2017). We analyzed Scopus publications retrieved with key words ‘‘Artificial intelligence’’. United States, China, Germany, United Kingdom and France were the most active in publishing research on artificial intelligence. Russia was in 30th place by the number of the publications related to artificial intelligence. China demonstrated the largest increase in the number of publications related to artificial intelligence during the last 17 years (Fig. 9) (Li 2017).

Discussion

Our analysis suggested some positive trends in research performance of the Russian institutions and high ranking of the Russian research in mathematics, physics and astronomy, engineering, chemistry, chemistry engineering, pharmacology, toxicology and 123 2034 Scientometrics (2018) 116:2019–2052 pharmaceutics. However, incremental improvements are small and cannot ensure leading position of the Russian science in the future. Russian government initiatives probably contributed to improving research performance among institutions (Turko et al. 2016; Tereshchenko and Shcherbakov 2017; Markusova et al. 2013). However, the lack of transparency in research funding and grant assessment resulted in the absence of positive association between research and development expenditure and incremental increase in the GDP (Schiermeier 2010). Our analyses are in concordance with previous publications suggesting some positive trends in scientific performance of the Russian universities (Poldin et al. 2017). Previously published comparative analyses with East European Universities suggested poor presen- tation of the Russian publications in internationals databases including Scopus and Web of Science (Kirillova and Soloshenko 2012). The Russian Index of Science Citation has been recognized as a valid characteristic of research performance in the Russian Federation (Nikolaev 2013; Galeev and Galeeva 2012; Tretyakova and Kabakov 2013; Sokolov 2014). However, we found no association of the Russian Index of Science Citation with internationally recognized measures of research performance. Harmonization between Scopus and Russian bibliographic databases would result in a better integration of the Russian research into international research and enrich scientific collaboration and progress. Routine comparative analysis of research performance of the Russian scientific institutions is needed to ensure optimal distribution of research funding and return on investment. Although we could not analyze exact return on investment in research and development in the Russian Federation in comparison with the US, the absolute value of research funding is much less in Russia then in the US. Therefore, overall cost of the research in mathematics, physics and other important areas is probably lower in Russia then in the US or other developed countries (Macilwain 2010; Lane and Bertuzzi 2011). Cost-effectiveness of Russian research specifically in areas of long-term excellence should be demonstrated to attract international investment. More active international collaboration with world leaders in various research areas may improve scientific and economic impact of Russian science (Ivanov et al. 2014; Aldieri et al. 2017; Frenken et al. 2017; Chen et al. 2017). Training in grant application should be a routine part of science education in Russia. Diversity in research performance among Russian universities should be reflected in state funding of specific institutions (Abankina et al. 2016; Lyagushkina and Bogatov 2017; The UK’s Department for Business 2017). Legal aspects of the Russian state policy should address national priorities in research and development in an open and competitive environment (Alexander and Magipervas 2015; Zemlin 2017). For instance, training and research in artificial intelligence should be supported at the state level (Meyer 2017). Finally, international media coverage of Russian scientific achievements would ensure full recognition, complete integration and better economic impact of the national research. Our analyses have several limitations. We relied on the available data and did not contact research institutions and specific researchers to validate analyzed metrics. We focused our analyses on the metrics available in Scopus and SciVal due to known defi- ciencies of other sources (Li et al. 2017; Aldieri et al. 2017). We compared research performance of the Russian universities with research institutions of the US and did not adjust analyses for some negative trends in research performance of the US universities. For instance, research performance of the universities in India and China demonstrate the fastest growth in many areas including artificial intelligence. All available metrics are

123 Scientometrics (2018) 116:2019–2052 2035 based on distribution of the available publications rather than valid measures of the direct scientific impact on people well-being (Lane and Bertuzzi 2011; Macilwain 2010). Despite these limitations, we explored the overall comparative trends in research activities of the Russian universities. Several factors may contribute to poor research performance of the Russian universities. First, difficulties in financial support and problematic administrative execution of the international research activities preclude a full integration of the Russian science into the world research activities. Legal aspects of the collaborative projects need to be developed to ensure equal rights, responsibilities and accountability of the Russian and foreign investors and researchers. Collaborative projects in social and political science are specifically problematic due to substantial differences in general legislation, political organization and barriers between the Russian Federation and other countries. In contrast with other countries, Russian scientific titles including books and journals are not available in English and are not included in the international bibliographic databases. Russian researches had to present their results in English language and published them in the foreign journals to be recognized by the international research community. The majority of science administrators in Russia do not encourage researchers to present their results in internationally recognized journals. Only few largest universities evaluate research performance of specific researchers with following financial and administrative rewards.

Conclusions and policy implication

We concluded that Russian research has a great potential despite much less ﬁnancial support at the national and international level. Routine analysis of the research performance and economic impact of R&D expenditure should be reﬂected in state funding of leading institutions and authors. Transparent distribution of research grants according to the research performance is necessary to ensure motivating rewards for the most productive researchers. Legal aspects of the international research must be developed to ensure a complete integration of the Russian science into international research activities. Compliance with ethical standards

Conflict of interest All authors declare that they have no conflict of interest.

Appendix

See Tables 6 and 7.

123 06Sinoerc 21)116:2019–2052 (2018) Scientometrics 2036 123 Table 6 Research performance and ranking of various institutions in the Russian Federation Institutions Publications Citations Citations per Publications Authors Field-Weighted Publicationa (growth %)a (growth %)a Citation Impacta

RAS 98,211 357,190 3.6 (52) 41.1 (143) 31.6 (1) 0.69 (105) Moscow State University 34,143 173,266 5.1 (28) 35.3 (149) 23.4 (2) 0.9 (58) RAS—Siberian Branch 25,296 98,940 3.9 (47) 43.7 (135) 22 (3) 0.73 (92) St. Petersburg State University 16,195 58,075 3.6 (53) 121.6 (90) 122 (5) 0.89 (60) Novosibirsk State Universityb 11,250 74,559 6.6 (18) 169 (73) 114.1 (9) 1.19 (30) Kazan Volga Region Federal Universityb 9974 30,323 3 (68) 425.7 (30) 327.2 (7) 1.13 (34) Moscow Engineering Physics Institute 9208 59,154 6.4 (20) 276.7 (44) 194.9 (10) 1.25 (26) Tomsk Polytechnic Universityb 8890 19,076 2.1 (110) 438.3 (28) 275.3 (8) 1.28 (22) Ural Federal Universityb 8248 18,289 2.2 (105) 181.7 (68) 146.7 (11) 0.65 (118) Joint Institute for Nuclear Research 7896 74,834 9.5 (12) 12.5 (173) 9.3 (17) 1.44 (16) Moscow Institute of Physics and Technologyb 7779 43,892 5.6 (23) 221.1 (52) 176 (15) 1.2 (29) Tomsk State Universityb 7776 26,157 3.4 (58) 574.9 (16) 271.1 (18) 1 (43) St. Petersburg National Research University of Information 7347 20,918 2.8 (76) 473.2 (23) 292.1 (14) 1.11 (36) Technologies, Mechanics and Optics (ITMO)b St. Petersburg State Polytechnical Universityb 6869 26,383 3.8 (48) 265.6 (45) 171 (12) 1.28 (21) Higher School of Economicsb 6703 22,386 3.3 (63) 293.3 (42) 260.4 (13) 1.25 (27) RAS—IoffePhysico- Technical Institute 6376 25,535 4 (44) 26.4 (157) 20.2 (23) 0.75 (82) RAS—Lebedev Physics Institute 5996 60,874 10.2 (10) 31.8 (150) 21.9 (28) 1.58 (15) Russian Ministry of Health 5861 34,128 5.8 (22) 221.2 (51) 181.1 (4) 1 (41) Ural Branch Russian Academy of Sciences 5636 14,191 2.5 (87) 53.8 (130) 48.5 (16) 0.52 (149) Russian Academy of Medical Sciences 5063 31,282 6.2 (21) - 43.8 (205) - 48.1 (6) 0.88 (61) Russian Research Centre Kurchatov Institute 4762 24,454 5.1 (29) 83.6 (109) 65.7 (19) 0.96 (48) National University of Science and Technology MISISb 4627 15,148 3.3 (64) 181.6 (69) 128.4 (24) 0.88 (62) Nizhni Novgorod State Universityb 4426 12,841 2.9 (73) 153 (80) 93.2 (22) 0.73 (93) Southern Federal University 4260 9892 2.3 (100) 146.3 (81) 138.4 (21) 0.72 (95) Table 6 continued 2037 116:2019–2052 (2018) Scientometrics

Institutions Publications Citations Citations per Publications Authors Field-Weighted Publicationa (growth %)a (growth %)a Citation Impacta

Alikhanov Institute for Theoretical and Experimental Physics 4066 67,881 16.7 (5) - 15.2 (199) - 0.6 (47) 2.25 (8) Far Eastern Federal Universityb 3870 7824 2 (114) 220.5 (53) 186 (30) 0.72 (96) RAS—Saint Petersburg Nuclear Physics Institute 3553 63,275 17.8 (3) 10.6 (176) 38 (46) 2.33 (7) RAS—Budker Institute of Nuclear Physics 3021 53,034 17.6 (4) 22.4 (161) 26.9 (45) 2.38 (6) Samara National Research Universityb 3007 8509 2.8 (77) 284 (43) 215.2 (34) 1 (44) Moscow State Technical University 2915 3807 1.3 (162) 254.1 (47) 200.3 (25) 0.52 (153) RAS—General Physics Institute 2844 10,209 3.6 (54) 37.9 (147) 9.2 (50) 0.67 (113) Sechenov First Moscow State Medical Universityb 2748 4022 1.5 (143) 296.8 (41) 244.4 (20) 0.37 (192) Peoples’ Friendship University of Russiab 2745 3899 1.4 (152) 226.8 (50) 204.8 (27) 0.69 (106) Institute for High Energy Physics 2692 69,136 25.7 (1) 0.4 (188) 1.3 (63) 3.08 (3) Siberian Federal Universityb 2633 6035 2.3 (101) 108 (93) 100.3 (29) 0.62 (127) Novosibirsk State Technical University 2455 4799 2 (115) 97.7 (100) 75.6 (32) 0.94 (50) RAS—Institute for Nuclear Researchb 2431 39,149 16.1 (6) 2.3 (185) - 1.3 (71) 2.23 (9) Saratov State University 2401 8073 3.4 (60) 62.7 (122) 32.7 (44) 0.74 (86) RAS—A. N. Nesmeyanov Institute of Organoelement Compounds 2386 10,210 4.3 (40) 3.2 (182) 17.6 (55) 0.65 (116) National Research University of Electronic Technology (MIET) 2219 3877 1.7 (129) 125.7 (89) 86.1 (33) 0.54 (140) RAS—Institute of Problems of Chemical Physics 2213 6596 3 (69) 43.7 (136) 24.9 (56) 0.5 (158) Pirogov Russian National Research Medical University 2192 7412 3.4 (59) 211.4 (57) 143.7 (26) 0.64 (123) RAS—Institute of Applied Physics 2167 12,239 5.6 (24) 19.3 (165) 26.4 (54) 1.4 (18) St. Petersburg State Electrotechnical University 2049 3403 1.7 (130) 325.2 (38) 222.2 (39) 0.86 (63) Boreskov Institute of Catalysis SB RAS 2022 10,984 5.4 (25) 46.2 (133) 18.7 (70) 0.82 (67) South Ural State Universityb 1956 3011 1.5 (144) 454.5 (25) 409.8 (37) 0.74 (88) 123 RAS—Sobolev Institute of Mathematics of SB 1949 3536 1.8 (124) 42.5 (139) 26.1 (105) 0.68 (112) RAS—Zelinsky Institute of Organic Chemistry 1937 9944 5.1 (30) 27.9 (154) 25 (61) 0.75 (80) RAS—Nikolaev Institute of Inorganic Chemistry of SB 1894 7951 4.2 (42) 11 (175) 4.5 (82) 0.69 (103) 08Sinoerc 21)116:2019–2052 (2018) Scientometrics 2038 123 Table 6 continued

Institutions Publications Citations Citations per Publications Authors Field-Weighted Publicationa (growth %)a (growth %)a Citation Impacta

RAS—Joint Institute for High Temperatures Research 1868 6909 3.7 (50) 56.9 (128) 34.8 (65) 0.8 (72) Voronezh State University 1792 3352 1.9 (119) 70 (118) 48.2 (35) 0.42 (182) RAS—Space Research Institute 1719 15,212 8.8 (13) 24.2 (160) 23.7 (85) 1.25 (25) RAS—Institute of Mineralogy and Petrography SB 1719 8736 5.1 (32) 42.7 (138) 30 (101) 1 (42) Belgorod State University 1698 4350 2.6 (83) 212.6 (56) 318.9 (31) 0.65 (117) Moscow Power Engineering Institute 1683 2375 1.4 (153) 143.3 (82) 124.1 (40) 0.53 (144) RAS—N.N. Semenov Institute of Chemical Physics 1632 4891 3 (70) 15.9 (169) 20.7 (66) 0.53 (142) Perm National Research Polytechnic University 1605 1797 1.1 (176) 203.2 (58) 201.2 (43) 0.4 (185) Volgograd State Technical University 1587 2079 1.3 (163) - 4.1 (193) 129.9 (48) 0.47 (169) RAS—P.P. Shirshov Institute of Oceanology 1567 5336 3.4 (61) 40.5 (145) 20.7 (72) 0.68 (109) RAS—Institute of Thermophysics, Siberian Branch 1518 4455 2.9 (74) 202.1 (59) 85.3 (102) 0.83 (66) RAS—Institute of Semiconductor Physics, Siberian Branch 1500 4932 3.3 (65) 42.2 (140) 27.8 (90) 0.61 (130) Mendeleev University of Chemical Technology 1498 3014 2 (116) 63.2 (121) 57.2 (36) 0.38 (188) Kazan National Research Technological University 1485 3104 2.1 (111) 141.5 (85) 93.8 (38) 0.91 (55) National Mineral Resources University 1440 1253 0.9 (188) 607 (13) 531.3 (53) 0.76 (78) Perm State National Research University 1379 3004 2.2 (106) 93.5 (104) 94 (52) 0.52 (151) RAS—Kirensky Institute of Physics, Siberian Branch 1377 5217 3.8 (49) 17.4 (167) - 1.4 (118) 0.74 (85) RAS—Steklov Mathematical Institute 1370 3085 2.3 (103) 58.2 (126) 68.4 (137) 0.93 (53) Moscow State Aviation Institute 1335 1558 1.2 (168) 128.9 (88) 101.4 (49) 0.46 (175) RAS—Institute for Atmospheric Optics SB 1291 5739 4.4 (37) 119.4 (91) 45 (127) 0.97 (46) Saratov State Technical University 1245 1696 1.4 (155) 109.7 (92) 73.7 (64) 0.43 (179) RAS—Shubnikov Institute of Crystallography 1222 3908 3.2 (66) 0 (189) - 1.6 (121) 0.56 (136) RAS—Institute of Solid State Physics 1169 5123 4.4 (38) - 7.6 (196) 7.7 (128) 0.75 (81) Moscow Polytechnic University 1163 1427 1.2 (169) 99.3 (98) 95.9 (51) 0.69 (107) RAS—Institute of Control Sciences 1137 1322 1.2 (172) 81.3 (112) 58.8 (111) 0.49 (161) Table 6 continued 2039 116:2019–2052 (2018) Scientometrics

Institutions Publications Citations Citations per Publications Authors Field-Weighted Publicationa (growth %)a (growth %)a Citation Impacta

RAS—Keldysh Institute of Applied Mathematics 1126 1986 1.8 (125) 40.9 (144) 41.3 (106) 0.53 (143) RAS—Institute for Problems in Mechanics 1117 1996 1.8 (127) 41.3 (141) 20.2 (143) 0.63 (126) Ivanovo State University of Chemical Technology 1080 2372 2.2 (107) 28.4 (153) 32 (81) 0.35 (196) Kazan State Technical University 1074 1586 1.5 (145) 235 (49) 174.2 (62) 0.94 (51) Bashkir State University 1074 2900 2.7 (82) 142.7 (83) 108.9 (60) 0.71 (98) RAS—Engelhardt Institute of Molecular Biology 1057 5394 5.1 (31) 94.4 (103) 57.3 (83) 0.73 (90) Plekhanov Russian University of Economics 1055 1296 1.2 (170) 581.5 (15) 673.7 (57) 1.15 (33) Samara State Technical University 1050 1783 1.7 (131) 238.8 (48) 184.4 (68) 0.77 (76) RAS—Institute of High Current Electronics SB 1036 2732 2.6 (86) 60.6 (124) 0.6 (140) 0.58 (134) Don State Technical University 980 1383 1.4 (154) 857.1 (9) 595.8 (58) 1.28 (23) Omsk State Technical University 963 894 0.9 (189) 403.9 (31) 317.6 (78) 0.71 (99) RAS—United Shmidt Institute of Physics of the Earth 960 2291 2.4 (95) 31.5 (151) 26.3 (117) 0.47 (167) Troﬁmuk Institute of Petroleum-Gas Geology and Geophysics of the 954 2526 2.6 (84) 38 (146) 31.9 (110) 0.75 (84) Siberian Branch of the RAS Reshetnev Siberian State Aerospace University 954 1672 1.8 (123) 135.5 (86) 105.6 (96) 0.74 (87) Tomsk State University of Control Systems and Radioelectronics 948 1500 1.6 (137) 161.3 (76) 92.1 (88) 0.77 (77) Ufa State Aviation Technical University 929 3281 3.5 (55) 96.3 (102) 102.4 (86) 0.92 (54) All-Russian Research Institute of Experimental Physics 915 6472 7.1 (17) 1.9 (186) - 8.8 (42) 0.95 (49) Yaroslavl State University 909 1261 1.4 (157) 57.4 (127) 68.1 (98) 0.65 (120) Landau Institute for Theoretical Physics 904 4439 4.9 (34) 24.3 (159) 17.7 (172) 1.02 (39) North-Eastern Federal University 900 2504 2.8 (78) 425.9 (29) 440 (67) 0.75 (83) Tyumen State Oil and Gas University 874 1355 1.6 (136) 2178.6 (2) 2088.9 (69) 1.19 (31) 123 I. M. Gubkin Russian State University of Oil and Gas 852 1247 1.5 (146) 79 (114) 50.3 (77) 0.47 (168) Russian Academy of Medical Sciences—N.N. Blokhin Russian 846 17,916 21.2 (2) 15.2 (171) 66.8 (59) 2.91 (4) Cancer Research Center Tyumen State Universityb 846 1603 1.9 (121) 960 (8) 737.5 (100) 1.08 (38) 00Sinoerc 21)116:2019–2052 (2018) Scientometrics 2040 123 Table 6 continued

Institutions Publications Citations Citations per Publications Authors Field-Weighted Publicationa (growth %)a (growth %)a Citation Impacta

Irkutsk State University 846 2146 2.5 (88) 96.8 (101) 46.7 (79) 0.7 (101) Southwest State University 840 1247 1.5 (147) 391.1 (33) 230.1 (80) 0.9 (59) RAS—Pushkov Institute of Terrestrial Magnetism, Ionosphere and 837 2866 3.4 (62) - 16.5 (200) - 17.1 (145) 0.78 (74) Radiowave Propagation Federal State Unitary Enterprise 824 806 1 (182) 7.1 (179) 0 (41) 0.27 (202) RAS—Lavrentyev Institute of Hydrodynamics SB 808 1609 2 (118) 81.8 (111) 54 (148) 0.68 (111) R.Y. Alekseev Nizhny Novgorod State Technical University 800 1370 1.7 (132) 158.1 (77) 190.4 (89) 0.46 (173) RAS—Pulkovo Astronomical Observatory 797 4156 5.2 (26) 29.8 (152) 13 (164) 0.64 (121) RAS—Institute of Cytology 795 3411 4.3 (41) 10.6 (177) - 5.8 (114) 0.64 (122) St. Petersburg National Research University Academic of the Russian 784 8326 10.6 (9) 80.4 (113) 16.4 (134) 1.28 (20) Academy of Sciences Kotel’nikov Institute of Radio Engineering and Electronics of Russian 782 2006 2.6 (85) 1.5 (187) - 26.7 (113) 0.52 (148) Academy of Sciences St. Petersburg State Institute of Technology 782 1957 2.5 (89) 35.7 (148) 21.6 (84) 0.52 (150) RAS—Saint Petersburg Department of Steklov Institute of 780 3419 4.4 (39) 50 (132) 40 (170) 0.78 (73) Mathematics Moscow State Pedagogical University 773 1455 1.9 (120) 67 (120) 52.1 (97) 0.6 (132) Vladimir State University 769 881 1.1 (178) 87.2 (106) 111.8 (103) 0.49 (162) RAS—Institute of Marine Biology 759 2125 2.8 (81) 13.9 (172) 20.2 (154) 0.56 (137) Immanuel Kant Baltic Federal Universityb 749 1707 2.3 (102) 676 (11) 688.5 (122) 0.78 (75) RAS—Special Astrophysical Observatory 719 8888 12.4 (8) 59.6 (125) 5.3 (168) 1.64 (14) RAS—Institute of Automation and Electrometry of SB 709 2601 3.7 (51) 17.1 (168) 40.8 (153) 0.69 (104) Russian Presidential Academy of National Economy and Public 699 570 0.8 (196) 1294.1 (4) 1084.2 (95) 0.76 (79) Administration Altai State University 674 1187 1.8 (126) 503.4 (19) 256.8 (125) 0.63 (125) RAS—Institute of Molecular Genetics 669 5277 7.9 (15) - 3.7 (192) 4.1 (109) 0.99 (45) Table 6 continued 2041 116:2019–2052 (2018) Scientometrics

Institutions Publications Citations Citations per Publications Authors Field-Weighted Publicationa (growth %)a (growth %)a Citation Impacta

North-Caucasus Federal University 660 820 1.2 (171) 392.7 (32) 403 (76) 0.51 (157) Kuban State University 657 1635 2.5 (90) 82.8 (110) 116.5 (99) 0.64 (124) Penza State University 644 964 1.5 (148) 166.7 (74) 239.7 (107) 1.25 (28) Herzen Russian State Pedagogical University 635 977 1.5 (149) 77.8 (116) 69.1 (115) 0.39 (187) RAS—Pavlov Institute of Physiology 618 1824 3 (71) - 4.5 (194) - 3.4 (135) 0.39 (186) Ul’Yanovsk State University 611 1441 2.4 (96) 85.7 (107) 76.2 (130) 0.93 (52) RAS—Institute of Solar-Terrestrial Physics SB 610 1975 3.2 (67) 25.3 (158) 27.3 (159) 0.46 (171) Ogarev Mordovia State University 607 826 1.4 (156) 385.7 (34) 251 (75) 0.72 (97) Nosov Magnitogorsk State Technical University 602 661 1.1 (177) 545.5 (17) 590.6 (74) 0.74 (89) Petrozavodsk State University 593 1200 2 (117) 195.3 (60) 215.1 (119) 0.91 (56) Siberian State Medical University 589 1278 2.2 (108) 163.5 (75) 116.2 (93) 0.49 (160) Russian Academy of Medical Sciences—Institute of Biomedical 587 2454 4.2 (43) 26.5 (156) 35 (139) 0.73 (91) Chemistry Russian Academy of Medical Sciences—Siberian Branch 573 2802 4.9 (33) - 58.1 (208) - 56.8 (87) 0.68 (108) Belgorod State Technological University 558 559 1 (183) 1030.8 (6) 1164 (73) 0.32 (201) Ufa State Petroleum Technological University 553 518 0.9 (190) 308.6 (39) 393.3 (92) 0.48 (164) Voronezh State Technical University 538 776 1.4 (158) 220.5 (54) 255.9 (104) 0.46 (174) Moscow State University of Design and Technology 537 218 0.4 (212) - 12.3 (198) - 13.6 (94) 0.38 (191) Central Aerohydrodynamic Institute (TsAGI) 531 780 1.5 (150) 45.1 (134) 54.2 (123) 0.81 (70) S.I. Vavilov State Optical Institute 515 1268 2.5 (94) 55.6 (129) 11.3 (156) 0.7 (102) Russian Academy of Medical Sciences—Institute of Experimental 512 1273 2.5 (92) 68.6 (119) 45.2 (126) 0.46 (172) Medicine

123 Tomsk State Pedagogical University 511 3723 7.3 (16) 100 (94) 84.8 (179) 1.11 (35) Volgograd State University 511 564 1.1 (179) 78.9 (115) 166 (124) 0.36 (195) State Research Center of Russian Federation—Institute of Physics and 498 1255 2.5 (91) - 18.2 (202) 7.4 (112) 0.51 (156) Power Engineering 02Sinoerc 21)116:2019–2052 (2018) Scientometrics 2042 123 Table 6 continued

Institutions Publications Citations Citations per Publications Authors Field-Weighted Publicationa (growth %)a (growth %)a Citation Impacta

Northern Arctic Federal University 492 436 0.9 (191) 183.8 (67) 254.8 (120) 0.38 (190) Tver State University 484 697 1.4 (159) 184.1 (66) 104.2 (142) 0.37 (194) Karelian Research Centre, Russian Academy of Sciences 458 849 1.9 (122) 195 (61) 163.2 (141) 0.66 (114) N L Dukhov All-Russian Scientiﬁc Research Institute of Automatics 455 1148 2.5 (93) 463.2 (24) 438.5 (150) 0.97 (47) Karpov Institute of Physical Chemistry 448 1086 2.4 (97) - 21.6 (203) - 43.6 (161) 0.4 (184) Kemerovo State University 445 929 2.1 (112) 177.5 (70) 112.9 (138) 0.45 (177) Vyatka State University 434 526 1.2 (173) 479.2 (21) 394.7 (116) 1.66 (13) Tula State University 431 578 1.3 (164) 97.9 (99) 63.9 (129) 0.47 (170) Kostroma State Technological University 429 345 0.8 (197) 22.4 (162) 5.5 (131) 0.43 (180) Russian Academy of Agricultural Sciences 397 1129 2.8 (79) - 67.4 (209) - 59.4 (91) 0.42 (181) RAS—Institute of Gene Biology 396 1830 4.6 (35) 20.6 (163) - 2.6 (158) 0.61 (129) Northern State Medical Academy 395 838 2.1 (113) 3.1 (183) - 5.7 (149) 0.54 (139) RAS—Institute of Astronomy 386 1995 5.2 (27) - 7.7 (197) 23.9 (182) 0.65 (115) Russian State Agrarian University, Moscow Timiryazev Agricultural 378 590 1.6 (138) 194.1 (62) 247.7 (132) 0.52 (152) Academy Russian State University For The Humanities – RSUH 369 246 0.7 (200) 381.8 (35) 385 (162) 0.54 (141) Omsk State University 367 536 1.5 (151) 83.7 (108) 75.5 (152) 0.37 (193) Troitsk Institute for Innovation and Fusion Research 364 1035 2.8 (80) 18 (166) 0 (146) 0.59 (133) Lukoil 351 137 0.4 (213) 61.9 (123) 118.6 (108) 0.15 (210) Gazprom 343 377 1.1 (180) 41.3 (142) 19.4 (133) 0.35 (198) Togliatti State University 338 1180 3.5 (56) 157.1 (78) 125 (165) 0.82 (68) Mari State University 311 397 1.3 (165) 453.3 (26) 509.1 (136) 1.01 (40) Bochvar All-Russian Scientiﬁc Research Institute of Inorganic 310 501 1.6 (139) 43.5 (137) 32.2 (147) 0.35 (197) Materials RAS—Kazan Physical Technical Institute 306 887 2.9 (75) 12 (174) 34.9 (173) 0.6 (131) Table 6 continued 2043 116:2019–2052 (2018) Scientometrics

Institutions Publications Citations Citations per Publications Authors Field-Weighted Publicationa (growth %)a (growth %)a Citation Impacta

Moscow State Institute of International Relations (MGIMO- 305 191 0.6 (205) 212.9 (55) 283.3 (166) 0.42 (183) University) Russian State Hydrometeorological University 284 1142 4 (46) 27.8 (155) 19.5 (169) 1.29 (19) Surgut State University 279 215 0.8 (198) 15.8 (170) - 3.3 (157) 0.48 (165) RAS—Timiryazev Institute of Plant Physiology 269 1075 4 (45) 3 (184) 28.1 (160) 0.51 (155) Orel State University named after IS Turgenev 262 247 0.9 (192) 438.5 (27) 589.5 (144) 0.85 (65) Buryatskogo State University 256 332 1.3 (166) 370.6 (36) 388.9 (167) 0.53 (145) Khlopin Radium Institute 233 521 2.2 (109) 9.8 (178) - 13.6 (176) 0.5 (159) Yaroslav-the-Wise Novgorod State University 226 394 1.7 (133) 190.5 (63) 393.8 (171) 0.65 (119) RAS—USSR Cardiology Research Center 220 3513 16 (7) - 32 (204) - 32.8 (155) 3.73 (2) North-Western State Medical University named after I.I. Mechnikov 199 1292 6.5 (19) - 50 (206) - 28.4 (151) 0.8 (71) Central Research Institute for Machine Building 197 318 1.6 (140) 51.7 (131) 68.2 (163) 0.44 (178) European University at St Petersburg 196 312 1.6 (141) 187.5 (65) 245.5 (185) 0.82 (69) The St. Petersburg State University of Economics 195 175 0.9 (193) 1160 (5) 1180 (174) 0.73 (94) Innopolis University 180 211 1.2 (174) NA NA 1.44 (17) Syktyvkar State University 172 412 2.4 (98) 357.1 (37) 310 (184) 0.45 (176) RAS—Institute of Spectroscopy 167 1378 8.3 (14) - 79.2 (210) - 70.7 (196) 0.9 (57) Moscow State University of Food Production 161 723 4.5 (36) 142.1 (84) 109.1 (178) 1.1 (37) Yugra State University 143 248 1.7 (134) 153.3 (79) 122.2 (186) 0.86 (64) Russian Ilizarov Scientiﬁc Center for Restorative Traumatology and 142 192 1.4 (160) 72.2 (117) 170 (180) 0.32 (200) Orthopedics Perm State Humanitarian Pedagogical University 136 150 1.1 (181) 262.5 (46) 76.5 (183) 0.38 (189)

123 Rostov State University of Economics 124 223 1.8 (128) 3300 (1) 5000 (177) 5.07 (1) All-Russian Scientiﬁc Research Institute of Atomic Reactors 124 432 3.5 (57) - 50 (207) - 53.5 (175) 0.52 (147) Cherepovets State University 102 89 0.9 (194) 92.3 (105) 272.7 (193) 0.57 (135) Moscow State University of Geodesy and Cartography 98 238 2.4 (99) 520 (18) 900 (192) 0.68 (110) 04Sinoerc 21)116:2019–2052 (2018) Scientometrics 2044 123 Table 6 continued

Institutions Publications Citations Citations per Publications Authors Field-Weighted Publicationa (growth %)a (growth %)a Citation Impacta

Sochi State University 97 161 1.7 (135) 188.9 (64) 146.2 (195) 1.79 (12) Pyatigorsk State Linguistic University 96 44 0.5 (208) NA NA 0.27 (203) Ulyanovsk State Pedagogical University 96 64 0.7 (201) - 4.5 (195) 70 (197) 0.14 (213) Kalmyk State University 88 36 0.4 (214) 500 (20) 633.3 (188) 0.1 (216) Murmansk State Technical University 87 137 1.6 (142) 7.1 (180) 43.8 (194) 0.62 (128) Ukhta State Technical University 87 43 0.5 (209) 600 (14) 540 (191) 0.23 (207) Saint-Petersburg Pasteur Institute 82 794 9.7 (11) 7.1 (181) NA 1.16 (32) Pskov State University 82 44 0.5 (207) 100 (95) 1000 (187) 0.56 (138) Moscow State Academy of Veterinary Medicine and Biotechnology 78 74 0.9 (195) 1000 (7) 188.9 (198) 0.52 (154) named after K.I. Skryabin Ulyanovsk State Agricultural Academy 74 168 2.3 (104) 1700 (3) 1566.7 (189) 1.86 (10) Krasnoyarsk State Agrarian University 61 58 1 (184) 475 (22) 380 (201) 0.24 (204) Moscow State Linguistic University – MSLU 51 66 1.3 (167) 650 (12) 450 (206) 1.27 (24) Russian New University 49 58 1.2 (175) 20 (164) 62.5 (205) 0.2 (208) Murmansk Arctic State University 44 14 0.3 (216) 133.3 (87) 30 (204) 0.53 (146) Moscow City Teachers’ Training 41 57 1.4 (161) 175 (71) 160 (202) 2.48 (5) University(euoyapdaybeyaayukbqcrov Moscow City Pedagogical University) Russian State University of Physical Education, Sport, Youth and 40 21 0.5 (210) 175 (72) 150 (199) 0.48 (166) Tourism The Gorbatov’s All-Russian Meat Research Institute 40 19 0.5 (211) NA NA 0.16 (209) Advanced Energy Technologies Ltd 26 77 3 (72) 300 (40) 800 (208) 0.48 (163) Moscow State Institute of Culture 25 18 0.7 (202) - 16.7 (201) 0 (215) 0.35 (199) Innovation and Technology Centre Proryv 20 21 1 (185) NA NA 0.23 (206) Vyatka State Agricultural Academy 19 5 0.3 (217) NA NA 0.24 (205) Novosibirsk State Agrarian University 18 15 0.8 (199) 700 (10) 700 (213) 0.71 (100) Table 6 continued 2045 116:2019–2052 (2018) Scientometrics

Institutions Publications Citations Citations per Publications Authors Field-Weighted Publicationa (growth %)a (growth %)a Citation Impacta

Saratov State Law Academy 14 4 0.3 (218) NA NA 1.86 (11) AtomEnergoproekt Ltd 14 5 0.4 (215) 100 (96) 400 (209) 0.08 (217) All-Russian Scientiﬁc Research Institute of Technical Physics & 12 1 0.1 (221) 100 (97) 800 (210) 0.01 (221) Automation Churapcha State Institute of Physical Culture and Sport 11 0 0 (222) NA NA 0 (222) Voronezh Institute of High Technologies 10 7 0.7 (204) NA NA 0.05 (218) Kuban State Agrarian University 7 5 0.7 (203) 0 (190) - 50 (216) 0.11 (215) Caspian Fisheries Research Institute, Russia 6 2 0.3 (219) - 100 (211) - 100 (218) 0.02 (220) Synergy university (Moscow University of Industry and Finance) 5 3 0.6 (206) NA NA 0.14 (214) Scientiﬁc Research Institute of the Tyre Industry, Moscow 3 1 0.3 (220) NA NA 0.05 (219) International Independent Ecological-Political University 2 2 1 (186) NA NA 0.14 (211) Institute of Nuclear Materials, Zarechnyi 2 0 0 (223) NA NA 0 (223) Buryat State Academy of Agriculture 1 1 1 (187) NA NA 0.14 (212)

a()- ranking among all Russian institutions bLeading Russian universities participating in the Program ‘‘5–top 100’’ 123 2046 Scientometrics (2018) 116:2019–2052

Table 7 Research performance and ranking of various institutions in the Russian Federation according to the Russian electronic library Institutions Russian Index of Publications Ratio of Science Citations citations/ publicationb

Institute for High Energy Physicsa 135,700 4448 30.5 Landau Institute for Theoretical Physics 75,593 3723 20.3 RAS—Institute of Astronomy 31,573 2039 15.5 Joint Institute for Nuclear Research 299,597 21,498 13.9 RAS—Engelhardt Institute of Molecular Biology 55,654 4118 13.5 RAS—Institute of Molecular Genetics 21,732 1698 12.8 RAS—Special Astrophysical Observatory 29,901 2453 12.2 RAS—Institute of Spectroscopy 34,014 3047 11.2 RAS—Timiryazev Institute of Plant Physiology 20,424 1859 11.0 RAS—USSR Cardiology Research Centera 68,328 6561 10.4 RAS—Institute of Solid State Physics 77,812 7480 10.4 St. Petersburg National Research University 4999 501 10.0 Academic of the Russian Academy of Sciences Saint-Petersburg Pasteur Institute 7906 824 9.6 RAS—Institute of Gene Biology 13,843 1485 9.3 RAS—Shubnikov Institute of Crystallography 102,457 11,132 9.2 Russian Academy of Medical Sciences—Institute of 22,980 2631 8.7 Biomedical Chemistry Russian Academy of Agricultural Sciences 8579 1034 8.3 RAS—Institute of Cytology 24,286 3002 8.1 Russian Academy of Medical Sciences 2742 339 8.1 Russian Academy of Medical Sciences—N.N. 70,138 8965 7.8 Blokhin Russian Cancer Research Centera Russian New University 90,129 11,745 7.7 RAS—Pulkovo Astronomical Observatory 16,434 2176 7.6 RAS—Institute of Mineralogy and Petrography, SB 49,794 6614 7.5 Tomsk State Pedagogical University 69,551 9397 7.4 RAS—P.P. Shirshov Institute of Oceanology 57,934 7855 7.4 RAS—Keldysh Institute of Applied Mathematics 86,258 11,885 7.3 RAS—Institute of Automation and Electrometry of 21,586 3010 7.2 SB Troitsk Institute for Innovation and Fusion Research 23,058 3304 7.0 RAS—N.N. Semenov Institute of Chemical Physics 65,801 9580 6.9 RAS—Institute of Applied Physics 71,410 10,558 6.8 Moscow State University of Geodesy and 30,543 4525 6.7 Cartography RAS—Institute of Semiconductor Physics, Siberian 38,345 5943 6.5 Branch Khlopin Radium Institute 6027 939 6.4 RAS—Institute for Problems in Mechanics 45,914 7578 6.1 S.I. Vavilov State Optical Institute 21,962 3640 6.0 RAS—Saint Petersburg Department of Steklov 17,160 2858 6.0 Institute of Mathematics 123 Scientometrics (2018) 116:2019–2052 2047

Table 7 continued

Institutions Russian Index of Publications Ratio of Science Citations citations/ publicationb

RAS—United Shmidt Institute of Physics of the 38,874 6531 6.0 Earth Trofimuk Institute of Petroleum-Gas Geology and 53,399 9097 5.9 Geophysics of the Siberian Branch of the RAS Moscow State University 1,005,960 171,429 5.9 Karpov Institute of Physical Chemistry 28,318 4917 5.8 RAS—Institute of High Current Electronics SB 20,911 3764 5.6 Voronezh Institute of High Technologies 12,804 2341 5.5 All-Russian Research Institute of Experimental 18,766 3440 5.5 Physics RAS—Institute for Atmospheric Optics SB 24,924 4580 5.4 European University at St Petersburg 10,793 2109 5.1 Russian State Agrarian University, Moscow 113,220 22,681 5.0 Timiryazev Agricultural Academy St. Petersburg State University 648,837 131,077 5.0 Kotel’nikov Institute of Radio Engineering and 47,997 9851 4.9 Electronics of Russian Academy of Sciences Moscow Engineering Physics Institute 137,039 29,598 4.6 RAS—Institute of Thermophysics, Siberian Branch 18,394 4012 4.6 Russian Academy of Medical Sciences—Institute of 28,353 6314 4.5 Experimental Medicine RAS—Pushkov Institute of Terrestrial Magnetism, 31,868 7105 4.5 Ionosphere and Radiowave Propagation Bochvar All-Russian Scientific Research Institute of 3199 716 4.5 Inorganic Materials Sechenov First Moscow State Medical University 298,983 67,179 4.5 RAS—Pavlov Institute of Physiology 11,782 2687 4.4 Moscow State Institute of International Relations 122,173 28,187 4.3 (MGIMO-University) RAS—Lavrentyev Institute of Hydrodynamics SB 17,506 4058 4.3 All-Russian Scientific Research Institute of 3729 893 4.2 Technical Physics & Automation Siberian State Medical University 56,527 13,556 4.2 RAS—Institute of Solar-Terrestrial Physics SB 13,316 3380 3.9 RAS—Institute of Control Sciences 33,065 8996 3.7 Ulyanovsk State Agricultural Academy 61,922 17,199 3.6 Ufa State Aviation Technical University 41,176 11,641 3.5 Tomsk State University 197,622 56,754 3.5 International Independent Ecological-Political 1766 509 3.5 University Pirogov Russian National Research Medical 73,149 21,147 3.5 University Higher School of Economics 145,597 42,974 3.4 Moscow State Academy of Veterinary Medicine and 31,109 9202 3.4 Biotechnology named after K.I. Skryabin

123 2048 Scientometrics (2018) 116:2019–2052

Table 7 continued

Institutions Russian Index of Publications Ratio of Science Citations citations/ publicationb

Moscow State Technical University 180,380 56,138 3.2 Kuban State Agrarian University 103,171 32,327 3.2 Pyatigorsk State Linguistic University 22,981 7433 3.1 I. M. Gubkin Russian State University of Oil and 39,699 13,015 3.1 Gas Institute of Nuclear Materials, Zarechnyi 785 258 3.0 Synergy University 6182 2052 3.0 Plekhanov Russian University of Economics 225,637 75,170 3.0 Ivanovo State University of Chemical Technology 28,600 9588 3.0 St. Petersburg State Polytechnical University 108,574 36,962 2.9 The St. Petersburg State University of Economics 115,939 39,486 2.9 Samara National Research University 94,252 32,340 2.9 Siberian Federal University 135,468 47,918 2.8 Ural Federal University 176,498 63,150 2.8 Mendeleev University of Chemical Technology 47,490 17,091 2.8 Southern Federal University 169,152 61,931 2.7 Novosibirsk State Technical University 106,353 39,280 2.7 Saratov State University 79,041 29,355 2.7 Yaroslavl State University 27,755 10,533 2.6 Kemerovo State University 53,400 20,397 2.6 Altai State University 74,005 28,356 2.6 St. Petersburg State Institute of Technology 16,103 6265 2.6 RAS 9220 3599 2.6 St. Petersburg State Electrotechnical University 26,900 10,787 2.5 Surgut State University 27,520 11,036 2.5 Russian State University For The Humanities— 143,164 58,236 2.5 RSUH Irkutsk State University 62,197 25,473 2.4 Moscow State Pedagogical University 58,504 24,041 2.4 Togliatti State University 34,066 14,012 2.4 Russian Presidential Academy of National Economy 210,605 88,086 2.4 and Public Administration Voronezh State Technical University 75,039 31,411 2.4 Peoples’ Friendship University of Russia 146,605 63,031 2.3 Vyatka State Agricultural Academy 12,821 5596 2.3 Russian State University of Physical Education, 14,827 6478 2.3 Sport, Youth and Tourism North-Western State Medical University named after 26,693 11,794 2.3 I.I. Mechnikov Murmansk State Technical University 21,193 9419 2.3 Perm State Humanitarian Pedagogical University 21,818 9842 2.2 Southwest State University 62,903 28,558 2.2 Ulyanovsk State Pedagogical University 23,474 10,682 2.2

123 Scientometrics (2018) 116:2019–2052 2049

Table 7 continued

Institutions Russian Index of Publications Ratio of Science Citations citations/ publicationb

Omsk State University 39,897 18,177 2.2 Syktyvkar State University 14,921 6808 2.2 Herzen Russian State Pedagogical University 161,723 73,914 2.2 Samara State Technical University 61,920 28,394 2.2 Tyumen State University 55,005 25,562 2.2 National University of Science and Technology 52,453 24,729 2.1 MISIS Moscow City Teachers’ Training University 52,962 25,018 2.1 Volgograd State University 39,711 18,774 2.1 Kazan Volga Region Federal University 107,954 51,084 2.1 Belgorod State Technological University 41,528 20,064 2.1 RAS—Institute of Marine Biology 4491 2178 2.1 Sochi State University 8920 4339 2.1 Voronezh State University 154,099 75,087 2.1 Nosov Magnitogorsk State Technical University 49,528 24,274 2.0 Kazan National Research Technological University 48,295 23,733 2.0 Russian State Hydrometeorological University 7340 3701 2.0 Karelian Research Centre, Russian Academy of 164 83 2.0 Sciences Moscow State University of Food Production 20,395 10,409 2.0 North-Caucasus Federal University 83,689 43,558 1.9 Tula State University 26,641 13,903 1.9 Kuban State University 62,455 33,540 1.9 Yaroslav-the-Wise Novgorod State University 11,829 6371 1.9 Ufa State Petroleum Technological University 75,989 41,570 1.8 Volgograd State Technical University 77,028 42,260 1.8 Caspian Fisheries Research Institute, Russia 959 528 1.8 Yugra State University 7871 4342 1.8 R.Y. Alekseev Nizhny Novgorod State Technical 19,272 10,633 1.8 University Saratov State Technical University 42,636 23,563 1.8 Ogarev Mordovia State University 68,139 37,690 1.8 Tomsk Polytechnic University 76,790 43,243 1.8 Novosibirskij Gosudarstvennyj Agrarnyj Universitet 14,162 8168 1.7 Tomsk State University of Control Systems and 14,913 8680 1.7 Radioelectronics Far Eastern Federal University 45,352 26,785 1.7 Penza State University 47,895 28,349 1.7 Kalmyk State University 16,428 9726 1.7 Tyumen State Oil and Gas University 58,498 34,872 1.7 Petrozavodsk State University 17,959 10,870 1.7 Ul’Yanovsk State University 20,737 12,609 1.6 Central Research Institute for Machine Building 3466 2121 1.6 123 2050 Scientometrics (2018) 116:2019–2052

Table 7 continued

Institutions Russian Index of Publications Ratio of Science Citations citations/ publicationb

Don State Technical University 46,969 29,356 1.6 Moscow Power Engineering Institute 22,742 14,418 1.6 Saratov State Law Academy 14,280 9141 1.6 Central Aerohydrodynamic Institute (TsAGI) 10,420 6708 1.6 Rostov State University of Economics 28,443 18,423 1.5 Perm State National Research University 29,607 19,271 1.5 South Ural State University 49,193 32,229 1.5 Moscow State Linguistic University—MSLU 11,998 7923 1.5 Perm National Research Polytechnic University 35,783 23,781 1.5 Kazan State Technical University 18,526 12,398 1.5 Immanuel Kant Baltic Federal University 13,328 9088 1.5 Russian Ilizarov Scientiﬁc Center for Restorative 3182 2197 1.4 Traumatology and Orthopaedics Moscow State University of Design and Technology 13,347 9218 1.4 Tver State University 20,714 14,629 1.4 Mari State University 24,009 17,883 1.3 Lukoil 601 448 1.3 Bashkir State University 49,217 37,772 1.3 Orel State University named after IS Turgenev 22,752 17,622 1.3 Moscow Polytechnic University 23,732 18,627 1.3 Belgorod State University 28,869 23,690 1.2 National Mineral Resources University 13,601 11,215 1.2 Gazprom 2986 2531 1.2 Moscow State Institute of Culture 5659 4879 1.2 Vladimirskij Gosudarstvennyj Universitet 12,840 11,550 1.1 Northern Arctic Federal University 18,302 16,836 1.1 Pskov State University 4272 3949 1.1 Buryatskogo State University 14,858 14,033 1.1 The Gorbatov’s All-Russian Meat Research Institute 18,109 17,275 1.0 Omsk State Technical University 25,305 24,793 1.0 North-Eastern Federal University 17,795 19,125 0.9 Vyatka State University 14,386 16,157 0.9 Ukhta State Technical University 3815 4635 0.8 N L Dukhov All-Russian Scientiﬁc Research 179 231 0.8 Institute of Automatics Krasnoyarsk State Agrarian University 5687 7828 0.7 Buryat State Academy of Agriculture 1455 2081 0.7 Murmansk Arctic State University 1049 1830 0.6 Reshetnev Siberian State Aerospace University 834 1508 0.6 Cherepovets State University 3186 6070 0.5 Innopolis University 15 29 0.5 Novosibirsk State University 106,353 392,804 0.3

123 Scientometrics (2018) 116:2019–2052 2051

Table 7 continued

Institutions Russian Index of Publications Ratio of Science Citations citations/ publicationb

Churapcha State Institute of Physical Culture and 69 283 0.2 Sport aUniversities with the highest ranking according to international indices bSorted from highest to the lowest

References

Abankina, I., Aleskerov, F., Belousova, V., Gokhberg, L., Kiselgof, S., Petrushchenko, V., et al. (2016). From equality to diversity: Classifying Russian universities in a performance oriented system. Tech- nological Forecasting and Social Change, 103(Supplement C), 228–239. https://doi.org/10.1016/j. techfore.2015.10.007. Aldieri, L., Kotsemir, M., & Vinci, C. P. (2017). The impact of research collaboration on academic performance: An empirical analysis for some European countries. Socio-Economic Planning Sciences. https://doi.org/10.1016/j.seps.2017.05.003. Alexander, C., & Magipervas, A. (2015). Features of the Advancement of Science as an Integral Part of the National Innovation System in Modern Russia. Procedia—Social and Behavioral Sciences, 166(Supplement C), 480–487. https://doi.org/10.1016/j.sbspro.2014.12.559. Asheim, B., Grillitsch, M., & Trippl, M. (2016). Smart Specialization as an innovation-driven strategy for economic diversification: Examples from Scandinavian regions. Advances in the theory and practice of smart specialization. Amsterdam: Elsevier Science Publishers (im Druck). Google Scholar. Bolotov, V., Kvelidze-Kuznetsova, N., Laptev, V., & Morozova, S. (2014). The h-Index in the Russian Science Citation Index. Educational Studies, 1, 241–262. Chen, K., Zhang, Y., Zhu, G., & Mu, R. (2017). Do research institutes benefit from their network positions in research collaboration networks with industries or/and universities? Technovation. https://doi.org/ 10.1016/j.technovation.2017.10.005. Colledge, L., & Verlinde, R. (2014). Scival metrics guidebook. Amsterdam: Elsevier. Decree of the Government of the Russian Federation. (2013). On measures of state support to the leading universities of the Russian Federation in order to increase their competitiveness among the world’s leading research and educational centers. http://www.consultant.ru/document/cons_doc_LAW_ 143479/, No. 211(March 16 (Edited on 15.11.2017)). Decree of the President of the Russian Federation. (2012). On measures to implement state policy in the field of education and science. http://www.consultant.ru/document/cons_doc_LAW_129346/, N 599(May 7). Federal Assembly of the Russian Federation., C. o. t. S. D. o. t. b. a. t. (2017). The Project of the Federal Law No. 15455-7 ‘‘On the Federal Budget for 2017 and on the plan period 2018-2019’’. December(N 15455-7), http://www.consultant.ru/cons/cgi/online.cgi?req=doc;base=PRJ;n=158072#158070. Frenken, K., Heimeriks, G. J., & Hoekman, J. (2017). What drives university research performance? An analysis using the CWTS Leiden Ranking data. Journal of Informetrics, 11(3), 859–872. https://doi. org/10.1016/j.joi.2017.06.006. Galeev, I. H., & Galeeva, N. H. (2012). Experience in analyzing publication and publishing activity using Russian Index of Science citations. Educational Technologies and Society, 15(1), 594–608. Gorin, S. V., Koroleva, A. M., & Ovcharenko, N. A. (2016). The Russian Science Citation Index (RSCI) as a new trend in scientific editing and publishing in Russia. European Science Editing, 42(3), 60–63. Ivanov, V., Libkind, A., & Markusova, V. (2014). Publication activity and research cooperation between higher education institutions and the Russian Academy of Sciences. Herald of the Russian Academy of Sciences, 84(1), 28–34. Kirillova, O. V., Soloshenko, NS., (2012). Comaprative analysis of the publication and citation activity between the Russian Federation and the countries of Eastern Europe. Educational Studies (1), 148–175. Kulmala, M., Lappalainen, H., Peta¨ja¨, T., Kurten, T., Kerminen, V.-M., Viisanen, Y., et al. (2015). Intro- duction: The Pan-Eurasian Experiment (PEEX)—Multidisciplinary, multiscale and multicomponent research and capacity-building initiative. Atmospheric Chemistry and Physics, 15(22), 13085–13096. Lane, J., & Bertuzzi, S. (2011). Measuring the results of science investments. Science, 331(6018), 678–680.

123 2052 Scientometrics (2018) 116:2019–2052

Li, X. (2011). Sources of external technology, absorptive capacity, and innovation capability in Chinese state-owned high-tech enterprises. World Development, 39(7), 1240–1248. Li, F.-.F (2017). Opening the Google AI China Center. https://www.blog.google/topics/google-asia/google- ai-china-center/, December 13. Li, J., Wu, D., Li, J., & Li, M. (2017). A comparison of 17 article-level bibliometric indicators of insti- tutional research productivity: Evidence from the information management literature of China. In- formation Processing and Management, 53(5), 1156–1170. https://doi.org/10.1016/j.ipm.2017.05.002. Lyagushkina, E., Bogatov, VV., (2017). Analysis of the effectiveness of fundamental scientific research: the publication aspect. Inter-Net Journal http://naukovedenie.ru,9(2 (39)), https://naukovedenie.ru/PDF/ 10EVN217.pdf. Macilwain, C. (2010). What science is really worth: spending on science is one of the best ways to generate jobs and economic growth, say research advocates. But as Colin Macilwain reports, the evidence behind such claims is patchy. Nature, 465(7299), 682–685. Markusova, V., Libkind, A., Mindeli, L., & Jansz, M. (2013). Research performance by Federal and National Research Universities and impact of competitive funding on their publication activity. Collnet Journal of Scientometrics and Information Management, 7(2), 217–229. Meyer D. (2017). Vladimir Putin Says Whoever Leads in Artificial Intelligence Will Rule the World. Fortune, September 4 (http://fortune.com/2017/09/04/ai-artificial-intelligence-putin-rule-world/). Mongeon, P., & Paul-Hus, A. (2016). The journal coverage of Web of Science and Scopus: A comparative analysis. Scientometrics, 106(1), 213–228. Nikolaev, V. (2013). Tools for assessing publication activity: Ways to improve. Bulletin of the Saratov State Law Academy, 5(94), 6–7. O’meara, S. (2015). At the very heart of progress. Nature, 528(7582), S179–S181. Pislyakov, V., & Shukshina, E. (2014). Measuring excellence in Russia: Highly cited papers, leading institutions, patterns of national and international collaboration. Journal of the Association for Infor- mation Science and Technology, 65(11), 2321–2330. Poldin, O., Matveeva, N. N., Sterligov, I. A., & Yudkevich, M. M. (2017). Publication activity of universities: the effect of the project ‘‘5-100’’. Educational Studies, 2, 10–35. Reznik-Zellen, R. (2016). Benchmarking with SciVal in Scholarly Communication and Research Services. Russian Academy of Science. (2013). The concept of development of the Russian Academy of Sciences until 2025. Meetings of the Presidium of the Russian Academy of Sciences, http://www.ras.ru/news/ shownews.aspx?id=ebd03228-2fd1-4291-8af8-89f7c081547a. Salter, B., Zhou, Y., Datta, S., & Salter, C. (2016). Bioinformatics and the politics of innovation in the life sciences: science and the State in the United Kingdom, China, and India. Science, Technology and Human Values, 41(5), 793–826. Schiermeier, Q. (2010). Russia to boost university science. Nature Publishing Group. Sokolov, D. V. (2014). Publication activity as a scientometric indicator: Russian and International experience. Journal of Science. Innovation. Education, 15, 131–147. Tereshchenko, D., & Shcherbakov, V. S. (2017). Economic analysis of publication activity in Russia. The Economics of Education, 3, 123–135. The Directive of the Government of the Russian Federation. (2012). Plan of measures to develop and improve the competitiveness of the leading Russian universities among the world’s scientific and educational centers. http://www.consultant.ru/cons/cgi/online.cgi?req=doc&base=LAW&n= 195286&fld=134&dst=1000000001,0&rnd=0.925379851922461#0, October 29(No. 2006-R). The Directive of the Government of the Russian Federation. (2013). Council on increasing the competitiveness of leading universities of the Russian Federation among the world’s leading scientific and educational centers. http://www.consultant.ru/cons/cgi/online.cgi?req=doc;base=EXP;n=613313#0, April (N 529-p). The UK’s Department for Business, E. I. S. B. (2017). International comparative performance of the UK research base 2016. elsevier.com/research-intelligence. Tretyakova, O., & Kabakov, E. A. (2013). Possibilities and prospects for using citation indexes in assessing the performance of a scientific institution. Economic and Social Changes: Facts, Trends, Forecast, 6(30), 189–200. Turko, T., Bakhturin, G., Bagan, V., Poloskov, S., & Gudym, D. (2016). Influence of the program ‘‘5–top 100’’ on the publication activity of Russian universities. Scientometrics, 109(2), 769–782. https://doi. org/10.1007/s11192-016-2060-9. The Ministry of Education and Science of the Russian Federation. 5-100 Russian Academic Excellence Project. https://5top100.ru/universities/. Zemlin, A. I. (2017). Legal aspects of the assessment of the performance of research organizations of the Ministry of Defense of Russia. Russian Military Journal, 1, 149–157.

123