Comparison of Bibliographic Data Sources: Implications for the Robustness of University Rankings
RESEARCH ARTICLE Comparison of bibliographic data sources: Implications for the robustness of university rankings Chun-Kai (Karl) Huang , Cameron Neylon , Chloe Brookes-Kenworthy , Richard Hosking , Lucy Montgomery , Katie Wilson , and Alkim Ozaygen an open access journal Centre for Culture and Technology, Curtin University, Bentley 6102, Western Australia Keywords: bibliographic data, data quality, open access, OpenCitations, research evaluation, Downloaded from http://direct.mit.edu/qss/article-pdf/1/2/445/1885863/qss_a_00031.pdf by guest on 25 September 2021 university ranking, Unpaywall Citation: Huang, C.-K., Neylon, C., Brookes-Kenworthy, C., Hosking, R., ABSTRACT Montgomery, L., Wilson, K., & Ozaygen, A. (2020).Comparison of bibliographic Universities are increasingly evaluated on the basis of their outputs. These are often converted data sources: Implications for the robustness of university rankings. to simple and contested rankings with substantial implications for recruitment, income, and Quantitative Science Studies, 1(2), 445–478. https://doi.org/10.1162/ perceived prestige. Such evaluation usually relies on a single data source to define the set of qss_a_00031 outputs for a university. However, few studies have explored differences across data sources DOI: and their implications for metrics and rankings at the institutional scale. We address this gap https://doi.org/10.1162/qss_a_00031 by performing detailed bibliographic comparisons between Web of Science (WoS), Scopus, Supporting Information: and Microsoft Academic (MSA) at the institutional level and supplement this with a manual https://www.mitpressjournals.org/doi/ suppl/10.1162/qss_a_00031 analysis of 15 universities. We further construct two simple rankings based on citation count and open access status. Our results show that there are significant differences across data- Received: 29 August 2019 Accepted: 14 January 2020 bases.
[Show full text]