An assessment of South Africa’s research journals: impact factors, Eigenfactors and the structure of editorial boards

Androniki EM Pouris and Prof Anastassios Pouris

Journals are the main Researchers would like as well-known researchers particular year. Despite the to know where to publish and deans of faculties are continuous debate related vehicles for scholarly in order to maximise asked to assess particular to the validity of Garfield’s the exposure of their journals. Subsequently, journal for communication research and what to read the collected opinions the identification of the in order to keep abreast are aggregated and a journal’s standing, citation within the academic of developments in their relative statement can analysis has prevailed in community. As such, fields. Librarians would like be made. However, this the past (Vanclay, 2012; to have the most reputed approach has the same Bensman, 2012). assessments of journals journals available, and drawbacks as . research administrators Will the opinions remain Scientific publishing are of interest to a use journal assessments the same if the experts in South Africa has in their evaluations of were different? How can experienced a revolution number of stakeholders, academics for recruitment, an astronomer assess a during the last ten years. promotion and funding plant science journal? Are In 2000, the South African from scientists and purposes. Editors are there unbiased researchers government terminated interested to know the in scientifically small its direct financial support librarians to research relative performance of countries? to research journals and administrators, editors, their journal in comparison only the South African with competitor journals. The second approach is Journal of Science and policy analysts and Finally, policy makers have using citation analysis Water SA continued to to monitor the quality of to rank journals. receive financial support. policy makers, for a journals because they Citations are the formal An investigation in use published articles as acknowledgement of 2005 showed that the variety of reasons. A indicators of the research intellectual debt to termination of government system’s success. previously published involvement in the affairs study was conducted research. The impact factor of the journals generally In South Africa, the of a journal is measured by had a beneficial effect on that aims to identify the assessment of scholarly the frequency with which the impact factors of the performance of South and scientific journals is the average article in that journals. particularly important journal has been cited in a African journals that because higher education During 2006, the Academy institutions receive of Science of South Africa are indexed by Thomson financial support from (ASSAf), at the request of government for their the Department of Science Reuters’ Journal research activities. and Technology (DST), These institutions produced a new strategic Citations Reports. receive financial support framework for South according to the number Scientific Africa’s research journals. of publications their staff This strategic framework members produce and publishing recommends, among publish in predetermined others, the periodic peer journals. in South review of the country’s Africa has journals and a change in Two approaches, expert the publishing approach, opinion and citation experienced i.e. a move into an open- analysis, are used for access system. Finally, in the comparison and a revolution 2008, Thomson Reuters assessment of journals. during the substantially increased the In expert opinion coverage of South African assessments, experts such last ten years. journals.

87 ESSAYS . INNOVATE 10 2015 The number of journals two years is cited in the Fersht, 2009). The the journal. It should be indexed in the Science year of observation and Eigenfactor ranking clarified that the editorial Citation Index increased the number of articles this system is also based on boards are both indicators from 17 in 2002 to journal has published in citations, but it accounts of quality and inputs in 29 in 2009 – an increase these preceding two years. for differences in prestige the process of publishing of 70%. The coverage of Today, the journal impact among citing journals. a journal. For example, social sciences journals in factor, as estimated by Hence, citations from researchers are selective in the Social Sciences Index Thomson Reuters in the Nature or Cell are valued terms of how they spend showed an even more , is highly, as opposed to their time and would prefer substantial increase from one of the most important citations from third-tier to be associated with “top” four in 2002 to 16 in 2009 – indicators in evaluative journals with a narrower journals. On the other hand, a 400% increase. bibliometrics (Bornmann, readership. Another international researchers Neuhaus and Daniel, 2011). difference is that while introduce standards and Methodology It is used internationally by the impact factor of a approaches in the peer the scientific community, journal has a one-year review of the articles that Assessment and among other things, for census period and uses may improve the relevant comparison of journals journal assessments, the previous two years journals. according to their impact research grants, academic for the target window, the factors is a well-established subsidy purposes, and Eigenfactor metrics have a The performance of approach in scientometrics, hiring and promotion one-year census period and South African journals despite its limitations and decisions. use the previous five years shortcomings. The impact for the target window. Table 1 shows the impact factor of a journal is defined More recently, the factors, quartiles and as the quotient between Eigenfactor® approach has The composition of scientific categories of the number of times an been developed and used the editorial boards the South African journals article published in a for journal assessment is an indicator of the covered by the Journal journal over the previous (Bergstrom and West, 2008; internationalisation of Citation Reports during

 Table 1: Impact factors and quartiles of pre-existing South African journals in Thomson Reuters’ Journal Citation Reports Science Citation Index in 2002, 2009 and 2010

Journal Impact Impact Quartile Quartile Quartile Category factor 2002 factor 2009 2002 2009 2010 African Entomology 0.455 0.420 3 4 4 Entomology African Journal of Marine 0.754 1.520 3 2 3 Marine and Freshwater Science Biology African 0.516 0.746 3 3 2 Zoology Bothalia – African Biodiversity 0.358 0.242 2 4 3 Plant Science and Conservation† Journal of the Southern 0.052 0.216 4 4 4 Metallurgy and Metallurgical African Institute of Mining Engineering and Metallurgy 3 – Mining and Mineral Processing Journal of the South African 0.366 0.224 3 4 3 Veterinary Science Veterinary Association Onderstepoort Journal of 0.506 0.430 3 3 3 Veterinary Science Veterinary Research Ostrich 0.149 0.250 4 4 4 Ornithology South African Journal of 0.381 0.412 3 3 3 Agriculture, Dairy and Animal Animal Science Science South African Journal of 0.394 1.080 2 3 4 Plant Science Botany South African Journal of 0.265 0.429 4 4 4 Multidisciplinary Chemistry Chemistry South African Journal of 0.659 1.013 2/3 2 4 Geology Geology South African Journal of 0.7 0.506 2 3 2 Multidisciplinary Science Science South African Journal of 0.25 0.429 4 4 4 Surgery Surgery South African Journal of 0.224 0.562 4 4 4 Wildlife Research South African Medical Journal 1.019 1.325 2 2 2 General and Internal Medicine Water SA 0.481 0.911 3 3 3 Water Resources † Previous title was Bothalia

2015 INNOVATE 10 . ESSAYS 88 2002 for that year and for The examination of the time editor assisted by an according to a funding 2009 and 2010. The ranking impact factors indicates editorial board. formula in which one of the to quartiles has been that 12 journals increased components is the number undertaken to consider their impact factors. Table 2 shows the impact of research publications. the variation in citations However, these increases factors, quartiles and Universities currently among the various scientific were insufficient to move scientific categories of receive more than R120 000 disciplines. them into higher quartiles. South African journals that for each publication that were recently added to the their staff and students Of the 17 South African It seems that, as in the Journal Citation Reports. publish in qualifying journals in the Journal domain of journals, it With the exception of journals. Citation Reports, four has become more and African Invertebrates, which declined in terms of more difficult to compete is positioned in the second The increase in the number quartiles from 2002 to internationally. It is also quartile of the relevant of indexed South African 2010. Only one journal – interesting to note that journals with an impact journals during recent years African Journal of Marine South African Journal of factor of 1.216, all the undoubtedly increases Science – improved its Science – the country’s other journals fall within the country’s scientific performance and moved flagship journal – the fourth quartile of their visibility. A comparison of from the third to the exhibited a substantial categories. the journals’ performance second quartile. South drop in its impact factor during 2002, 2009 and 2010 African Journal of Geology and its position among Inclusion in the identified a relative decline moved definitively into the multidisciplinary journals international citation in impact factor. During second quartile in the list internationally during 2009. indices as an indicator the most recent period, the of the relevant disciplinary However, it recovered of journal visibility is majority of South African journals during 2009, during 2010 and was part crucial. In South Africa, indexed journals belong to while it was exactly in the of the second quartile inclusion in the citation the fourth and third quartile middle of the second and journals. It can be argued indices is particularly in terms of impact factor. third quartiles during 2002. that this variability is the important, as publications Only three of the scientific During 2010, this journal result of changes in the in the indexed journals journals were in the second moved into the fourth journal’s management automatically qualify the quartile. Similarly, all the quartile. The journal’s structure. Prior to 2009, country’s universities social sciences journals impact factor dropped from the journal had a full-time for government subsidy. were in the third and fourth 1.013 during 2009 to 0.638 editor. From 2009, it moved Universities in South Africa quartiles. Journals in the during 2010. to a model with a part- receive government subsidy tail of the Thomson Reuters

 Table 2: Impact factors and quartiles of South African journals newly added to Thomson Reuters’ Journal Citation Reports Science Citation Index

Journals Impact Quartile Quartile Category factor 2009 2009 2010 African Invertebrates 1.216 2 3 Zoology African Journal of Herpetology 0.455 4 4 Zoology African Journal of AIDS Research 0.569 4 4 Public, Environmental and Occupational Health International Sportmed Journal 0.171 4 4 Sport Science Journal of the South African Institution 0.125 4 4 Civil Engineering of Civil Engineering Quaestiones Mathematicae 0.267 4 4 Mathematics South African Journal of Enology and 0.314 4 3 Food Science and Technology Viticulture Southern African Journal of HIV 0.457 4 4 Infectious Diseases, Virology Medicine South African Journal of Industrial 0.093 4 4 Industrial Engineering Engineering South African Journal of Obstetrics 0 – 4 Obstetrics and Gynaecology South African Journal of Psychiatry 0.409 4 4 Psychiatry Southern Forests 0.5 4 3 Forestry

89 ESSAYS . INNOVATE 10 2015 ranking are at risk of being monitoring in countries References dropped from the citation like South Africa, which indices. Furthermore, monitors their science Bensman, JS. 2012. The impact factor: its place in Garfield’s thought, as researchers prefer to performance using the in science evaluation, and in submit their articles to Thomson Reuters indices. library collection management. high-impact journals, the The addition of journals in Scientometrics, 92(2):263–275. Bergstrom, C and West, JD. 2008. journals in the tail run the the indices increased the Assessing citations with risk of not receiving an coverage of the various Eigenfactor™ metrics. Neurology, adequate number of quality countries’ scientific articles, 71:1850–1851. Bornmann, L, Neuhaus, C and articles, and hence will but created discontinuities Daniel, H-D. 2011. The effects either reduce their quality The addition in the time series data. of a two stage publication standards or cease to exist. Similarly, the differentiated process on the journal impact factor: a case study on the of new coverage of the various interactive journal. The identification of the journals disciplines (for example, Atmospheric Chemistry and South African journals’ social sciences versus Physics. Scientometrics, 86(1): 93–97. Eigenfactors will provide engineering) can create in the Fersht, A. 2009. The most influential a valuable benchmark of changes in the countries’ journals: impact factor and performance for future indices has publication profiles. Science Eigenfactor. Proceedings of the National Academy of Sciences of investigations, as there is authorities should take implications the USA, 106(17):6883–6884. currently no historical data action to create compatible Thomson Reuters. 2008. The scientific of this indicator. for the future time series and relevant business of Thomson Reuters launches expanded journal country-scientific profiles. coverage in by The identification of the of science adding 700 regional journals structure of the editorial Finally, it should be [press release]. Available at policy. http://science. thomsonreuters. boards emphasises the mentioned that the addition com/press/2008/. above findings. The majority of South African journals (accessed 27 May 2008). of South African journals in the citation indices has Vanclay, KJ. 2012. Impact factor: outdated artefact or stepping- are dominated by local not adversely affected the stone to journal certification? researchers. As many as country’s scientific profile. Scientometrics. 92(2): 211–238. 20 journals do not have article, these ties impair Even though Thomson any foreign researchers or them from being objective. Reuters (2008) stated that academics on their editorial While other approaches “the importance of the boards. As international (such as monitoring the [inclusion of the] regional gatekeepers, they can comments of peers and journal is measured in transmit international increasing the number terms of the specificity of standards and practices in of members on editorial its content rather than in local journals, and because boards) may be able to its citation impact,” this they may increase the alleviate the challenges of investigation shows that the prestige of the journal bias, the incorporation of newly added journals were with their presence, the international researchers of the same quality in terms relevant authorities should on editorial boards is of impact factor as the pre- address the issue. It probably among the most existing ones. should be emphasised that effective approaches. international researchers on editorial boards alleviate It is important for the the shortcomings of peer prestige of the country, review in scientifically and of ASSAf, to take small countries like South appropriate actions to Africa. It has been argued improve the country’s that, in scientifically small journals. The approach countries, a small number of coupling scientometric of researchers work in assessments with peer the same field. They know reviews can further provide each other personally and evidence of the validity of are socially tied to each the above findings. other and to the social Prof Anastassios Pouris Androniki EM Pouris community surrounding The addition of new is associated with the is associated with the Faculty them. When these journals in the indices Institute for Technological of Science at the Tshwane researchers have to render has implications for the Innovation at the University University of Technology in a verdict on a research future of science policy and of Pretoria. Pretoria. proposal or research warrants intensive relevant

2015 INNOVATE 10 . ESSAYS 90