Wikipedia, a Social Pedia: Research Challenges and Opportunities: Papers from the 2015 ICWSM Workshop User Engagement on Wikipedia: A Review of Studies of Readers and Editors Marc Miquel Ribé Universitat Pompeu Fabra [email protected] Abstract on Wikipedia. Their results help in explaining Is it an encyclopedia or a social network? Without consider- how engagement occurs on both the reader and editor ing both aspects it would not be possible to understand how sides. Yet, there are some attributes presenting room for a worldwide army of editors created the largest online improvement detected almost five years ago; overall usa- knowledge repository. Wikipedia has a consistent set of bility and the design of particular communication channels rules and it responds to many of the User Engagement Framework attributes, and this is why it works. In this pa- could be revised in order to mitigate frustration. per, we identify these confirmed attributes as well as those The duality between the two groups of users, readers and presenting problems. We explain that although having a editors, has acted similarly to a feedback loop system; new strong editor base Wikipedia is finding it challenging to content availability helped to popularize the encyclopedia maintain this base or increase its size. In order to understand and improved the position for searchers, which in turn in- this, scholars have analyzed Wikipedia using current metrics like user session and activity. We conclude there ex- creased its use and its editing base in order to create new ist opportunities to analyze engagement in new aspects in articles. Suh et al. (2009) explains this growth as a self- order to understand its success, as well as to redesign mech- reinforcing mechanism, the more valuable Wikipedia be- anisms to improve the system and help the transition be- came the more contributors joined it and gave value to it. tween reader and editor. Although during the 2015 first quarter the number of edi- tors increased2, the general trend during the last few years 1. Introduction has been a soft decline. Ortega, Gonzalez-Barahona and Robles (2008) found that a very engaged minority of edi- Wikipedia has become the paradigm of collaborative crea- tors was responsible for most of the activity, and Stuart and tion success as well as an icon of Internet possibili- Halfaker (2013) verified that those who joined in 2006 are ties. Since 2001, it has grown to 4,5 M articles in the Eng- still the most active group. lish edition and 34 M in total counting the 288 languages Thus, questions like how readers become editors or how in which it is available. However, the most surprising is to raise writing activity have become relevant to the com- that this process has been made a reality by thousands of munity, Wikimedia Foundation and scholars (Okoli 2014). editors who have devoted their free time, converting it into Triggered by them, researchers applied some of the most a free product for mass consumption, while aiming at usual metrics in user experience, such as session analysis, “gathering the sum of all human knowledge". Moreover, it in order to analyze the different types of editors. Wikipedia is used all over the world use it, and this is confirmed by its is a very suitable object for analysis with longitudinal data 1 position in the top 10 Alexa rank of most visited sites. and every action performed tracked in its databases. Many researchers have tried to understand how the sys- The aim of this study is to put together all the stud- tem works, or in other words, what the pillars of its success ies from the perspective of readers and editors perspective are. User Engagement framework defines the attributes to give an integrated overview; we want to understand which constitute an engaging experience (O’Brien and the uses and difficulties users encounter during their expe- Toms 2008). Reliability, trust and expectation, richness riences. We want to see how they engage and disengage, as and control are some which have been studied by scholars stated by Attfield et al. (2011) in the definition “the rela- tionship they establish at a behavioral, emotional, and cog- nitive level”. Copyright © 2015, Association for the Advancement of Artificial Intelli- gence (www.aaai.org). All rights reserved. 1 http://www.alexa.com/topsites 2 https://meta.wikimedia.org/wiki/Research:Active_editor_spike_2015 67 We begin by examining studies concerning some aspects 2.2 User Engagement Framework of engagement in Wikipedia; then we review their attrib- utes and how they have been described. We assess the state User Engagement has been an evasive concept to define, of the community with its social and structural characteris- for it has been used to express qualities similar to attract- tics; we want to see its composition, current engagement ing, captivating or enticing depending on the study (Lal- and its growth possibilities for growth by attracting new mas and O’Brien 2014; Chapman, Selvarajah and Webster editors. Next we review all the available literature that ana- 1999). Some researchers grounded it to psychological theo- lyzes readers and editors, both by using metrics based on ries like Flow, Play and Aesthetics, which can explain dif- user activity, community and topics. Finally, we argue for ferent aspects of it (O’Brien and Toms 2008). Flow, for in- some improvements proposed to current research and out- stance, describes the state in which there is attention, con- line future engagement research. trol and an intrinsic interest (Csikszentmihalyi 1997). Many studies presented engagement as a quality of user experience with certain attributes that influence or 2. How Wikipedia engages compound it. O’Brien and Toms (2008), in their User En- gagement Framework, listed them as challenge, aesthetic, Wikipedia has been approached in numerous peer- sensory appeal, feedback, novelty, interactivity, aware- reviewed academic articles both as a data source and as a ness, motivation, interest and affect. Additionally, the du- study object to understand how it works (Okoli 2009). Alt- ration of the experience or its repetition has been consid- hough many facets of the encyclopedia have been studied ered a clear indicator of engagement (Attfield et al. 2011). profoundly, no research covers directly its quality of en- From the site manager perspective, creating engagement gagement as a whole, either for readers or editors, to un- is positive as it gives continuity to its users. Wikipedia derstand how a combination of factors contributes to its aims at creating high-quality information through the en- quality and repeatable experience. gagement of a broad and multilingual community. As a website its success depends on its qualities to captivate edi- 2.1 Wikipedia as an everyday tool tors as well as to give content to fulfill readers' expecta- tions. Engagement attributes like reputation and trust, nov- Engagement studies have been applied to many different elty and attention are more linked to the readers, while user contexts, from games and educational sites to e- context, motivation, usability and positive affect in addi- commerce and online news (O’Brien 2008). The broad ap- tion influence editors more directly. plication of the concept has brought very different out- I discuss each of the most common attributes in en- puts; literature has differentiated everyday engagement gagement research and their relation to Wikipedia studies. from games and other kinds in the sense that it is a less immersive experience (Lalmas and O’Brien Reputation, trust and expectation 2014). Wikipedia is this kind of engagement; the encyclo- Wikipedia’s reputation has always been questioned along pedia has become an object integrated into our everyday with its reliability. Readers need to know if the content work and personal lives. Wikipedia usage has been report- 3 they are reading is trustable; an Encyclopedia created by a ed in foundation studies as divided into different devices group of anonymous people initially seemed audacious and including phones, tablets and personal computers. Wikipe- doomed to failure. However, in 2006 a study compared it dians often multitask and edit Wikipedia while watching with the Encyclopedia Britannica and showed that it had TV, even chatting in IRC Wikipedia dedicated channels or fewer errors (Giles 2005). One study showed that the better any other social network. the coordination between editors, the higher the quality of Although editors and readers behave differently depend- the articles (Kittur and Kraut 2008). Wikipedia’s main ed- ing on the role they are taking, their behavior repeats over iting rule is to reach a Neutral Point of View (NPOV) with- (Stuart and Halfaker 2013). Surveys and analytical studies in each article. Instead of objectivity, a contrast of different identify some users who spend several hours a day. A usu- positions and their representation in the text is required to al behavior is to switch from article to article using their editors for an article to have quality. wiki hypertextual structure. Explorative navigation allows An issue threatening content quality is vandalism. It is us to frame Wikipedia use as an inter-site engagement, the confronted with policies and bots, which can restore old kind of experience in a network of pages in the same site content and ban the user who is misbehaving. Generally, (Yom-Tov et al. 2013). authors work on articles providing all the available data and references. Lucassen and Schraagen (2010) in a study developed the features by which an article is considered 3 http://upload.wikimedia.org/wikipedia/Editor_Survey_Report_- valuable (text, images, references). Likewise, the commu- _April_2011.pdf 68 nity considers which articles have quality characteristics of fun, ideology and community values were significant in a quality and list them in ‘featured articles’. In the end of non-reward scenario like Wikipedia.