Bibliometric Quality of Serbian Journals 2002-2011: More Than Just a Dress for Successi
Total Page:16
File Type:pdf, Size:1020Kb
Journal Publishing in Developing, Transition and Emerging Countries Proceedings of the 5th Belgrade International Open Access Conference 2012 Belgrade, Serbia, May 18-19, 2012 Edited by Pero Šipka Journal Publishing in Developing, Transition, and Emerging Countries Proceedings of the 5th Belgrade International Open Access Conference 2012 Belgrade, Serbia May 18-19, 2012 Pero Šipka (ed.) Publisher Centre for Evaluation in Education and Science Kneza Miloša 17 11.000 Belgrade Serbia tel. +381 11 32 38 506 www.ceon.rs Copyright information © 2013 Centre for Evaluation in Education and Science ISBN 978-86-89475-01-2 ISSN (print): 2334-7880 ISSN (online): 2334-7880 Cover by PeteF Ordering information Print-on-demand by Amazon. Web access and citation information This entire proceedings can also be viewed on the web at http://boac.ceon.rs/public/full/5th-bioac.pdf . Each paper has a unique identifier (DOI) which can be added to citations to facilitate access. The DOI # should not replace the full citation. Each e-version contains instruction on how to cite a paper. doi:10.5937/BIOAC-110 Bibliometric Quality of Serbian Journals 2002-2011: More Than Just a Dress for Successi Pero Šipka [email protected] Centre for Evaluation in Education and Science, Kneza Miloša 17, 11000 Belgrade (Serbia) Keywords: journals; bibliometrics; references; measuring; indicators; impact; citations; articles; Serbia Introduction tionalized by Šipka (2004). Maričić refer- Within a program sponsored by the gov- red to such indicators as “communicabili- ernment, Serbian journals are routinely ty”, which was not broadly accepted. For evaluated, ranked and “categorized”. Re- lack of consensus, in CEON/CEES prod- sults are published yearly in the Journal ucts they are dubbed “indicators of bib- Bibliometric Report (http://scindeks- liometric quality” (IBQs), while the term bic.ceon.rs/eng) by the Centre for Evalua- “indicators of impact” was kept for all tion in Education and Science (CEON/ measures based on the citation rate. Au- CEES; http://ceon.rs/). Journal ranking is thors studying the quality of journals pub- based primarily on impact, i.e. citations lished in developing countries (e.g. Faria received in both the Web of Science et al., 2007; Sheibaninia et al., 2010) find (WoS) and the Serbian Citation Index such indicators valid and useful. On the (SCIndeks; www.scindeks.ceon.rs). Im- contrary, those dealing with reputed in- pact indicators are supplemented with a dexed journals quite expectedly do not group of measures reflecting the formal find them useful and sometimes avoid characteristics of journals. These are de- them explicitly (e.g. Lokker et al., 2012). rived by quantifying ISI (now Thomson Reuters) qualitative admission criteria, such as the share of references in English, The purpose of the original introduction which can be generated with low-cost of IBQs in Serbia was to stimulate the from the SCIndeks data. These indicators process of standardization and interna- are supplemented with a few classical tionalization of national research publish- bibliometric measures, such as the jour- ing. At a practical level, the purpose was nal-to-monograph ratio, selected mostly to provide editors with feedback on how on the basis of findings about the quality close their journals are to the critical level of Yugoslav journals, later reviewed and of quality, sufficient for admission to systemized by Maričić (2002) and opera- WoS/JCR. Journal Publishing in Developing, Transition and Emerging Countries: Proceedings of the 5th Belgrade International Open Access Conference 2012, pages 161-166. Edited by P. Šipka Šipka Unlike the JBR Impact Factor, IBQs have (3) IntAuth: share (%) of articles by never been fully accepted by either deci- international authors; sion-makers or journal editors. Informal- ly, most of them referred to such indica- (4) IntLang: share (%) of articles publish- tors as outward, formal journal features, ed in English and other “world not having much to do with the real value languages”; of journals, or simply as “the suit that doesn't make the man”. (5) IntRefs share (%) of international references; and In this study, the results on the IBQs of (6) JourRef: share (%) of journal referen- Serbian WoS-indexed journals gathered ces (“journal-to-monograph ratio”); during the whole 10-year period of fol- lowing journals were observed in order to Repeated measures two-group ANOVA recognize the overall trends of their was used to test the differences in IBQs changes. The trends were compared with between WoS- and non-WoS-indexed those of non-WoS national journals. To journals. Only 17 WoS-indexed and 111 complete the picture, they were related to non-indexed journals with no missing da- the movement of journals’ WoS Impact ta were included in this part of the analy- Factor in the same period. The general sis. purpose was to re-evaluate, after a decade of their continuous use, the face validity The Impact Factor of the Serbian WoS- and benefitiality of IBQs. indexed journals was compared with jour- nals of the same status published in other Method SEE countries. The comparison covered the period from 2002 to 2011. All the da- Data for all journals (N=417), including ta were retrieved from JCR. In order to newly established and discontinued ones, include as many journals as possible, the were extracted from the JBR 2002-2011 Impact Factor calculated for a two year editions. All journals covered by WoS in period (IF2) was used. IF results were 2010 were treated as WoS-indexed jour- normalized by calculating the z-values nals (N=20), while the rest were allocated against the mean and S.D. of the group to to the group of non-WoS journals. which a journal was assigned in JCR. For journals classified in more than one JCR The two groups of journals were com- group, the z-value was calculated within pared by using the following IBQs: each group and each year separately. The annual score for each country was ex- (1) NumRef: average number of referen- pressed as the mean of all z-values of all ces per article; national journals. Due to the low cover- age of journals from most countries, big (2) AgeRef: average age of references variations of coverage within countries, (citing half-life); and relatively large number of countries 162 Bibliometric Quality of Serbian Journals 2002-2011: More Than Just a Dress for Success in the sample, using statistical tests in this variable showing a significant interaction case was not feasible. effect. Major differences seem to exist in the Share of journal references (JourRef), Results and Discussion as a measure of scientific seriosity, and Figure 1, displaying the results of Serbian Share of international references (IntRef), WoS-indexed journals, indicates a slow as a measure of internationality. To sum- but steady growth of IBQs during the ob- marize, WoS-indexed journals are gener- served period. The same tendency was ally superior to non-WoS journals, espe- registered for non-WoS journals (Figure cially on the measures of internationality. 2). Results given in Table 1 suggest that On most IBQs WoS-indexed journals did the only exception is the Share of interna- not reach a plateau, clearly indicating that tional references (IntRefs), which did not there is still some room for their improve- change significantly. This could not have ment. been expected, since this indicator al- ready reached the sub-maximal value in In Figure 3 all the y-axis values are nega- WoS-indexed journals almost at the very tive, revealing that regional journals as a beginning of the period followed. group are well below the world average. No sign of them nearing the world IF The two groups of journals differ signifi- means can be observed. All that can be cantly on all IBQs, except for Age of ref- seen as a general trend is that the impact erences (AgeRefs), which is also the only level of national journals became more 100.0 90.0 80.0 70.0 60.0 50.0 40.0 30.0 20.0 10.0 0.0 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 IntAuthors IntLanguage NumRefs JournRefs IntRefs AgeRefs Figure 1. IBQ results for WoS-indexed Serbian journals 163 Šipka 100.0 90.0 80.0 70.0 60.0 50.0 40.0 30.0 20.0 10.0 0.0 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 IntAuthors IntLanguage NumRefs JournRefs IntRefs AgeRefs Figure 2. IBQ results for non-WoS Serbian journals homogenous. Also, rankings changed Table 1. ANOVA results for IBQs of from year to year. The most remarkable Serbian journals change is the elevation of IF of Serbian WoS * journals. We see that during the period WoS years under study the previously discussed sta- years ble rise of the bibliometric quality of F p F p F p these journals was paralleled by the rise NumRef 9.30 .00* 11.6 .00* 0.56 .83 of their Impact Factor. Admittedly, this AgeRef .692 .41 3.87 .00* 1.97 .04* trend was abruptly terminated in 2008, IntAuth 18.22 .00* 5.72 .00* 1.45 .17 but this happened under the influence of an external factor, i.e. the extension of IntLang 18.76 .00* 3.88 .00* 1.03 .41 WoS coverage to the region, which is ex- IntRefs 25.78 .00* 1.57 .12 0.30 .98 plained in detail in another study (Kosa- JourRef 28.88 .00* 9.64 .00* 1.50 .14 nović & Šipka, in press). It should be pointed out that this trend was only bro- The increase of the impact of Serbian ken, not reversed, as was the case with journals was so rapid that it can even be most other SEE countries.