A Brief Review of Studies of Wikipedia in Peer-Reviewed Journals Chitu Okoli John Molson School of Business Concordia University, Montréal, Canada [email protected] Abstract the encyclopedia’s content under two conditions: the original source must be cited, and everyone else must Since its establishment in 2001, Wikipedia, “the be accorded the same right of free modification and free encyclopedia that anyone can edit” has become a distribution of all derivative works via the FDL. cultural icon of the unlimited possibilities of the World Similar to open source software, Wikipedia aims to Wide Web. Thus, it has become a serious subject of create high-quality digital information products scholarly study to objectively and rigorously through the participation of large numbers of understand it as a phenomenon. This paper reviews contributors, mostly volunteers, though some might be studies of Wikipedia that have been published in peer- paid. It is structured to eliminate or minimize reviewed journals. Among the wealth of studies individual agendas and strive towards public or reviewed, major sub-streams of research covered industry welfare in setting policies for the development include: how and why Wikipedia works; assessments of of the encyclopedia. Their organizational structures are the reliability of its content; using it as a data source generally rather loose, yet there is some central for various studies; and applications of Wikipedia in administration that permits the project to survive and different domains of endeavour. flourish. All along its history, anecdotal evidence has abounded as to Wikipedia’s quality, mainly by 1. Introduction referring to its popularity (it is regularly ranked by Alexa as a global top-ten website), and by noting the status of some of the organizations that regular cite it, Although the open source approach has traditionally such as the British Broadcasting Corporation. been applied only to software products, recent years However, more recently, Wikipedia has begun to have demonstrated its applicability to the creation of attract the attention of scholars who have attempted to other information products, most notably to the open more rigorously study the Wikipedia phenomenon content Wikipedia, “the free encyclopedia that anyone from many different angles. This study attempts to can edit” (www.wikipedia.org). In just seven years build a base for scholarly research on the subject of since its establishment in 2001, this comprehensive Wikipedia by reviewing most of the important general encyclopedia has compiled over ten million scholarly work that has been done thus far. Because the articles in 253 languages (with over 2.5 million in focus here is on high-quality scholarly work, the English). Other open content encyclopedias, though studies reviewed here are limited to those that have not nearly as well developed, include the Association been published in peer-reviewed journals. Thus, work for Information Systems’ ISPedia (http:// that has been published in conference proceedings is ispedia.terry.uga.edu), Enciclopedia Libre Universal en not included, nor has other work that has not passed Español (http://enciclopedia.us.es), Wikinfo (http:// through scholarly peer review. www.wikinfo.org), and Citizendium (http:// I begin by reviewing studies concerning how www.citizendium.org). Wikipedia works, and why it works successfully. Then Wikipedia is based on wiki technology [1], a social I examine a large body of research that uses various collaboration Web application that allows viewers to approaches, both comparative and epistemological, to add content to Web pages using minimal technical assess the reliability of Wikipedia. Then there are skills. The submitted content is published under the many studies that refer to, depend on, or focus on Free Software Foundation’s Free Documentation Wikipedia as a source for information or data in the License (FDL)—the textual complement to the General study. Finally, a few studies propose and examine Public License for open source software. This license permits anyone to freely copy, modify, and distribute 1 applications of Wikipedia and the Wikipedia concept for ideological commitment to the project. However, in education and economics. other hypothesized motivation categories such as social reasons, career advancement, and protection, were not 2. How and why Wikipedia works found to be very relevant. The earliest peer-reviewed articles on Wikipedia 2.3. The editorial process were more or less general introductory reviews, introducing readers to the novel idea of an A number of studies have gone further to try to encyclopedia that anyone can edit. More advanced understand the details of how the editorial process of studies began to dig deeper by investigating the Wikipedia operates to produce high-quality motivations of Wikipedians to contribute, and studying encyclopedia articles. Brandes and Lerner [12] the details of the process of developing articles. specifically examined the historical progress of edit wars, where controversies lead to mutual revisions of 2.1. Reviews of Wikipedia contributions. They presented visual analysis tools to study the progress of such conflicts. Okoli and Oh [13] Remy [2] provided perhaps the first review of studied how Wikipedians’ interactions in both direct Wikipedia in a peer-reviewed journal, just a year after article creation and in discussions with each other its establishment. She considered the free-editing affected their election to the status of “administrator,” concept dubious, and referred readers rather to which gives greater editorial privileges. Stvilia et al Nupedia, the expert-written online encyclopedia that [14] examined the article-creation process in preceded Wikipedia [3]. Wikipedia to discover how information quality is Hall [4] and Strategic Finance [5] provided reviews assured. They found that Wikipedia’s discussion pages that were generally neutral, mainly stressing the provide an extensive resource of documentation on novelty of the concept. Krause [6] briefly discussed how the processes of error detection and correction Wikipedia in the context of online social collaboration, occur. Den Besten and Dalle [15] studied the Simple as a prime example of wiki collaboration. McFedries English Wikipedia, a Wikipedia distinct from the [7] discussed wikis, referring to Wikipedia as the prime English one that limits its vocabulary sense and example. Morse [8] interviewed Jimmy Wales, the grammatical structure to facilitate reading by children founder of Wikipedia. Wales mainly discussed the and learners of English. They investigated the editorial workings and value of wikis for companies, and when process that implemented the rules instated to keep they were appropriate or inappropriate. Arter [9] articles “simple.” reviewed Wikipedia from the point of view of its peer- review mechanism as a means of ensuring quality. 3. The reliability of Wikipedia 2.2. Motivations for contributing So far, perhaps the most abundant body of scholarly work conducted on Wikipedia has been formal studies In the study of open source software, the question of evaluating the reliability of Wikipedia, variously why people code for free has always been a fascinating expressed as trustworthiness, quality, or accuracy. In subject of study. Similarly, a few studies have explored other words, why trust the contents of an encyclopedia why Wikipedians contribute their free time to the that anyone can edit? The most famous scholarly encyclopedia, especially considering that, unlike in the assessment of Wikipedia is a comparison of selected case of open source software, it is extremely rare for science articles in Wikipedia and Encyclopaedia anyone to be paid for their efforts. Britannica conducted by Nature journal [16]. The In their study of public contribution to Amazon study found Wikipedia’s accuracy comparable to those book reviews and to Wikipedia articles, Peddibhotla of Britannica. Among 42 articles, Wikipedia had an and Subramani [10] found multiple motives for average of four errors each, and Britannica had three. contribution to Wikipedia, both self-oriented and other- In this section I will first present a large number of oriented. Extending critical mass theory, they found negative assessments of Wikipedia’s reliability. Then I that the higher the quality of contributions, the lower will present a number of studies that take an epistemic their quantity, and vice versa. Moreover, the approach of assessing the reliability of Wikipedia; this motivations for contribution high-quality articles were perspective unequivocally considers the radical different from those for contributing a high quantity of encyclopedia in a very favourable light. articles. Nov [11] found that Wikipedians are motivated to contribute primarily for the fun of it, and 2 3.1. Criticisms Britannica. With such criteria, he argued that Wikipedia has been repeatedly shown to be quite Denning et al [17] question whether Wikipedia’s reliable [16]. Moreover, when compared to its more collaborative editing process is capable of producing likely alternate sources on the Web, Wikipedia is accurate and authoritative information on a thoroughly strikingly superior as a source of knowledge [26]. He comprehensive scope of human knowledge. Gorman argued that Wikipedia has important epistemological [18] contends that Wikipedia has no basis to call itself properties (“e.g., power, speed, and fecundity”) that an encyclopedia, and that without regulation of its