A Case Study of the Use of a Wiki As a Tool to Keep Systematic Reviews up to Date
Total Page:16
File Type:pdf, Size:1020Kb
Research Bender et al. Collaborative authoring: a case study of the use of a wiki as a tool to keep systematic reviews up to date JACQUELINE L BENDER, LAURA A O’GRADY, AMOL DESHPANDE, ANDREA A CORTINOIS, LUIS SAFFIE, DON HUSEREAU, ALEJANDRO R JADAD ABSTRACT Background: Systematic reviews are recognized as the most effective means of summarizing research evidence. However, they are limited by the time and effort required to keep them up to date. Wikis present a unique opportunity to facilitate collaboration among many authors. The purpose of this study was to examine the use of a wiki as an online collaborative tool for the updating of a type of systematic review known as a scoping review. Methods: An existing peer-reviewed scoping review on asynchronous telehealth was previously published on an open, publicly available wiki. Log file analysis, user questionnaires and content analysis were used to collect descrip- tive and evaluative data on the use of the site from 9 June 2009 to 10 April 2010. Blog postings from referring sites were also analyzed. Results: During the 10-month study period, there were a total of 1222 visits to the site, 3996 page views and 875 unique visitors from around the globe. Five unique visitors (0.6% of the total number of visitors) submitted a total of 6 contributions to the site: 3 contributions were made to the article itself, and 3 to the discussion pages. None of the contributions enhanced the evidence base of the scoping review. The commentary about the project in the blogosphere was positive, tempered with some skepticism. Interpretations: Despite the fact that wikis provide an easy-to-use, free and powerful means to edit information, fewer than 1% of visitors contributed content to the wiki. These results may be a function of limited interest in the topic area, the review methodology itself, lack of familiarity with the wiki, and the incentive structure of academic publishing. Controversial and timely topics in addition to incentives and organizational support for Web 2.0 impact metrics might motivate greater participation in online collaborative efforts to keep scientific knowledge up to date. Affiliations at the time of writing: Jacqueline L. Bender, MSc PhD (ABD) is a Doctoral Student at the Dalla Lana School of Public Health, Univer- sity of Toronto with the People, Health Equity and Innovation Research Group (Phi Group) at the Centre for Global eHealth Innovation, University of Toronto and University Health Network, Toronto, Ontario, Canada. Laura O’Grady, PhD, is a Postdoctoral Fellow with the People, Health Equity and Innovation Research Group (Phi Group) at the Centre for Global eHealth Innovation, University Health Network and the Department of Health Policy, Management and Evaluation, University of Toronto. Amol Deshpande, MD, MBA, is a Physician with the Comprehensive Pain Program, University Health Network, Toronto, Ontario, Canada. Andrea A. Cortinois, MPH, PhD, is a Research Associate with the People, Health Equity and Innovation Research Group (Phi Group) of the Centre for Global eHealth Innovation, University of Toronto and University Health Net- work, Toronto, Ontario, Canada. Luis Saffie, BSc, is a Systems Administrator/Technical Specialist with the Centre for Global eHealth Innovation, University of Toronto and University Health Network, Toronto, Ontario,Canada. Donald Husereau, BScPharm, MSc, is an Adjunct Professor in the Department of Epidemiology and Community Medicine, University of Ottawa, Ottawa, Ontario, Canada. Alejandro R Jadad, MD, DPhil, FRCPC, FCAHS is the Canada Research Chair in eHealth Innovation; Chief Innovator and Founder at the Centre for Global eHealth Innovation, University of Toronto and University Health Network; Professor at the Departments of Dalla Lana School of Public Health, Health Policy, Management and Evaluation, and Anesthesia, University of Toronto, Toronto, Ontario, Canada. Funding: This study was funded by CADTH, the Canadian Agency for Drugs and Technologies in Health. The funding agency did not influence the study design or the analysis and preparation of the manuscript. Competing interests: None declared. Correspondence: Jacqueline L. Bender, Centre for Global eHealth Innovation, R. Fraser Elliott Building, 4th floor, Toronto General Hospital, 190 Elizabeth St.,Toronto, ON M5G 2C4; [email protected] Open Medicine 2011;5(4)e201 Research Bender et al. YSTEMATIC REVIEWS HAVE BECOME THE MOST Wikicancer.org). Research is underway to explore the effective means of identifying, selecting, assessing utility of wikis as a knowledge base and a pedagogical Sand synthesizing all original research evidence tool.14,15 To the best of our knowledge, however, there relevant to a specific question. Most systematic reviews have been no published research efforts to explore the have finite life spans, with a median survival (time from role that wikis could play in relation to updating and publication until availability of new information that maintaining peer-reviewed systematic reviews of health potentially changes the effect size or direction) of 5.5 interventions. This study was designed to begin to ad- years; it has been estimated that, for 23% of systematic dress this gap. reviews, new information is available within 2 years of The purpose of this study was to examine the use publication.1 Despite the need to keep reviews current, of a wiki as an online collaborative tool for updating a only 17.7% of systematic reviews seem to be cited as up- peer-reviewed scoping review. A scoping review is a dated versions of previously published reviews.1, 2 type of systematic review that provides a comprehen- Given its open, transparent nature and the availabil- sive summary of the existing literature or evidence base ity of free software for online collaboration, the Internet on a broad topic.16,17 Similarly to traditional systematic presents a unique opportunity to explore new ways to reviews, scoping reviews follow rigorous and transpar- update and publish academic research. Wikis, in par- ent methods to identify all relevant literature.16 Unlike ticular, are efficient tools for co-creating, maintaining a full systematic review, however, they typically do not and making widely available repositories of knowledge. provide a quality assessment of included studies or ex- In general, a wiki allows anyone to easily add to, edit tensive data synthesis.17 A peer-reviewed scoping review or delete the content of a website. The wiki application was adapted for publication on an open, public wiki. This keeps track of all changes and offers the ability to ac- article describes the characteristics of the users, their knowledge author contributions. It has been suggested use of the site, and the nature, quality and quantity of that some of the benefits of using a wiki for collabora- the contributions, as well as the “buzz” that accumulated tive authoring include the ability to track authorship, on the Internet during the study period. monitor the development of an article, reduce conflicts of interest and update it swiftly, thus providing a current Methods evidence resource.3–5 The open, public wiki was created and made available Wikis are used very successfully as highly efficient through the Open Medicine web site from 2 June 2009 tools to co-create and maintain large amounts of gen- to 10 April 2010, under the name Open Medicine wiki eral-interest content. A case in point is Wikipedia, the (http://wikisr.openmedicine.ca/index.php/Main_Page). largest encyclopedia in the world. With over 3.5 mil- The wiki was populated with a pre-existing, peer- lion pages of articles in English alone, each of which is reviewed scoping review on asynchronous telehealth by edited 19 times on average, and over 13 million regis- Deshpande and colleagues18 that was initially published tered users,6 this free resource overtook all other en- by CADTH in print and online form in January 2008, cyclopedias in size7 and possibly even the Encyclopedia after peer review.19 In preparation for its publication for- Britannica in quality, in less than 5 years.8 Wikipedia, mat on the Open Medicine wiki, the review underwent which contains over 20 000 health-related articles, has a second round of peer review by the editorial board of become a popular health resource for the general pub- Open Medicine. lic and the medical community.5,9 The top 200 medical The Open Medicine wiki utilized the open-source articles on Wikipedia receive, on average, over 100 000 platform developed by Mediawiki (the same software page views per month,10 and studies suggest that 50% to application used by Wikipedia) to create a viewing and 70% of practising physicians use Wikipedia as a resource navigating experience that would likely be familiar to when providing care.5,11,12 users of the site (Fig. 1). The wiki was hosted and admin- Although Wikipedia is by far the largest and most istered by the Centre for Global eHealth Innovation, a popular collaboratively authored reference website, joint research initiative of the University of Toronto and there are now over 70 health-related wikis in existence the University Health Network in Toronto. It was brand- of varying specialties and depth.13 Some serve as a gen- ed with the Open Medicine logo and linked to the Open eral medical reference (e.g. Ganfyd.org) or subspecialty Medicine website. resource (e.g. Wikisurgery.org) for health professionals, Initially, the Open Medicine wiki was launched as a while others are intended as a shared knowledge base for public, “unrestricted” wiki open for anyone to view, add, people with specific health concerns or conditions (e.g. change or delete content, and join. Registration was not Open Medicine 2011;5(4)e202 Research Bender et al. required to contribute or modify content on the site, in • bounce rate (frequency of times a visitor lands on an effort to lower any perceived barriers to participation.