Journal.Pone.0115253
Total Page:16
File Type:pdf, Size:1020Kb
Edinburgh Research Explorer Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot Citation for published version: Klein, M, Van de Sompel, H, Sanderson, R, Shankar, H, Balakireva, L, Zhou, K & Tobin, R 2014, 'Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot', PLoS ONE, vol. 9, no. 12, e115253. https://doi.org/10.1371/journal.pone.0115253 Digital Object Identifier (DOI): 10.1371/journal.pone.0115253 Link: Link to publication record in Edinburgh Research Explorer Document Version: Publisher's PDF, also known as Version of record Published In: PLoS ONE General rights Copyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s) and / or other copyright owners and it is a condition of accessing these publications that users recognise and abide by the legal requirements associated with these rights. Take down policy The University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorer content complies with UK legislation. If you believe that the public display of this file breaches copyright please contact [email protected] providing details, and we will remove access to the work immediately and investigate your claim. Download date: 07. Oct. 2021 RESEARCH ARTICLE Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot Martin Klein1*, Herbert Van de Sompel1, Robert Sanderson1, Harihar Shankar1, Lyudmila Balakireva1, Ke Zhou2, Richard Tobin2 1. Digital Library Research and Prototyping Team, Research Library, Los Alamos National Laboratory, Los Alamos, New Mexico, United States of America, 2. Language Technology Group, The University of OPEN ACCESS Edinburgh, Edinburgh, Scotland, United Kingdom Citation: Klein M, Van de Sompel H, Sanderson R, *[email protected] Shankar H, Balakireva L, et al. (2014) Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot. PLoS ONE 9(12): e115253. doi:10.1371/journal.pone.0115253 Editor: Judit Bar-Ilan, Bar-Ilan University, Israel Abstract Received: August 13, 2014 The emergence of the web has fundamentally affected most aspects of information Accepted: November 20, 2014 communication, including scholarly communication. The immediacy that Published: December 26, 2014 characterizes publishing information to the web, as well as accessing it, allows for a Copyright: ß 2014 Klein et al. This is an open- dramatic increase in the speed of dissemination of scholarly knowledge. But, the access article distributed under the terms of the Creative Commons Attribution License, which transition from a paper-based to a web-based scholarly communication system also permits unrestricted use, distribution, and repro- duction in any medium, provided the original author poses challenges. In this paper, we focus on reference rot, the combination of link and source are credited. rot and content drift to which references to web resources included in Science, Data Availability: The authors confirm that, for Technology, and Medicine (STM) articles are subject. We investigate the extent to approved reasons, some access restrictions apply to the data underlying the findings. XML files which which reference rot impacts the ability to revisit the web context that surrounds STM summarize the articles in our corpora are available articles some time after their publication. We do so on the basis of a vast collection via Figshare repository. Articles from arXiv are available at: http://dx.doi.org/10.6084/m9.figshare. of articles from three corpora that span publication years 1997 to 2012. For over 1132671; articles from Elsevier are available at: one million references to web resources extracted from over 3.5 million articles, we http://dx.doi.org/10.6084/m9.figshare.1132677; http://dx.doi.org/10.6084/m9.figshare.1132676.As determine whether the HTTP URI is still responsive on the live web and whether per Elsevier TDM policy, the files can only be re- used for non-commercial purposes. Articles from web archives contain an archived snapshot representative of the state the PMC are available at: http://dx.doi.org/10.6084/m9. referenced resource had at the time it was referenced. We observe that the fraction figshare.1132674.Thelivewebtestdatais available at: http://dx.doi.org/10.6084/m9.figshare. of articles containing references to web resources is growing steadily over time. We 1132678. The Memento data for each corpus is find one out of five STM articles suffering from reference rot, meaning it is also available via Figshare. Data from arXiv are available at: http://dx.doi.org/10.6084/m9.figshare. impossible to revisit the web context that surrounds them some time after their 1132672. Data from Elsevier are available at: http:// publication. When only considering STM articles that contain references to web dx.doi.org/10.6084/m9.figshare.1132675. Data from PMC are available at: http://dx.doi.org/10.6084/m9. resources, this fraction increases to seven out of ten. We suggest that, in order to figshare.1132673. safeguard the long-term integrity of the web-based scholarly record, robust Funding: The Hiberlink project is funded by the Andrew W. Mellon Foundation under the name solutions to combat the reference rot problem are required. In conclusion, we "Time Travel for the Scholarly Web." The funders provide a brief insight into the directions that are explored with this regard in the had no role in study design, data collection and analysis, decision to publish, or preparation of the context of the Hiberlink project. manuscript. Competing Interests: The authors have declared that no competing interests exist. PLOS ONE | DOI:10.1371/journal.pone.0115253 December 26, 2014 1/39 Scholarly Context Not Found Introduction Reference Rot in Web-Based Scholarly Communication Referencing sources is a fundamental part of the scholarly discourse. There is an expectation that referenced sources can and should be checked by others, to allow a correct interpretation of information that is being communicated and to support reproducibility of results. This credo continues as scholarly commu- nication transitions from being a paper-based to a web-based endeavor. But, as research communication, and increasingly also the research process, transition to the web, the range of scholarly assets that are being communicated and referenced is greatly increasing. In the paper-based era, a scholarly article would typically reference other scholarly articles. In the web-based era, the scope of referencing crucially still includes articles but has extended to also cover a wide range of assets that are used or created during the research process such as software, ontologies, scientific workflows, datasets, online debates, presentations, blogs, videos, etc. Also, whereas references in the paper-based era were purely textual, in the web era they additionally include HTTP URIs - from here on referred to as URIs - that provide convenient and immediate access to referenced resources on the web. This immediacy is one of the web’s transformative characteristics that is inherited by web-based scholarly communication and that allows for a dramatic increase in the speed of knowledge dissemination. But web-based scholarly communication also inherits some of the rather frustrating characteristics of the web, and, in this paper, we focus on one: reference rot. Reference rot is a term we introduced in the Hiberlink project [1] to denote the combination of two problems involved in using URI references, both of which relate to the dynamic and ephemeral nature of the web: N Link rot: The resource identified by a URI may cease to exist and hence a URI reference to that resource will no longer provide access to referenced content. N Content drift: The resource identified by a URI may change over time and hence, the content at the end of the URI may evolve, even to such an extent that it ceases to be representative of the content that was originally referenced. Link rot is known to all web users as they are regularly confronted with unhelpful ‘‘404 Not Found’’ error messages. Content drift, while very real, features less prominently as an annoyance of the web, probably because it rarely manifests itself during a browsing session but rather over an extended period of time. Both link rot and content drift pose a threat to the long-term persistence and integrity of the new-era scholarly record. Indeed, as references rot or as the content they originally referred to changes, it becomes impossible to revisit the intellectual context that surrounded the referencing article at the time of its publication. This is a significant step backwards when compared to the paper-based journal era: revisiting the context in those days was always possible even though it may have PLOS ONE | DOI:10.1371/journal.pone.0115253 December 26, 2014 2/39 Scholarly Context Not Found required visiting several libraries, each one a hub in a global, distributed archive for the journal literature. This threat was recognized early on when journal articles found their way onto the web. In order to combat link rot, the Digital Object Identifier (DOI) was introduced to persistently identify journal articles. In addition, the DOI resolver [2] for the URI version of DOIs was introduced to ensure that web links pointing at these articles remain actionable, even when the articles change web location. A similar approach has meanwhile emerged for research data, which is increasingly regarded as an integral part of the scholarly record. Content drift is hardly a matter of concern for references to journal articles, because of the inherent fixity that, especially PDF-formated, articles exhibit. Nevertheless, special-purpose solutions for long-term digital archiving of the digital journal literature, such as LOCKSS [3], CLOCKSS [4], and Portico [5], have emerged to ensure that articles and the articles they reference can be revisited even if the portals that host them vanish from the web.