Identifying, Understanding and Detecting Recurring, Harmful Behavior Patterns in Collaborative Wikipedia Editing – Doctoral Proposal –

Identifying, Understanding and Detecting Recurring, Harmful Behavior Patterns in Collaborative Wikipedia Editing – Doctoral Proposal – Fabian Flöck Institute AIFB Karlsruhe Institute of Technology Germany fabian.fl[email protected] Supervised by Dr. Elena Simperl, University of Southampton, UK and Dr. Achim Rettinger, Karlsruhe Institute of Technology, Germany ABSTRACT together orders of magnitude more users than their offline counter- In this doctoral proposal, we describe an approach to identify recur- parts. These environments thrive and prosper from the contribu- ring, collective behavioral mechanisms in the collaborative interactions of their user base, and their success hinges greatly on the ex- tions of Wikipedia editors that have the potential to undermine the tent to which interaction and collaboration among their users leads ideals of quality, neutrality and completeness of article content. We to a critical mass of useful, high-quality content. outline how we plan to parametrize these patterns in order to un- One of the most prominent examples of online collaborative plat- derstand their emergence and evolution and measure their effective forms in the past decade is undoubtedly Wikipedia. It is the di- impact on content production in Wikipedia. On top of these results rect result of the actions of a community of volunteers who create we intend to build end-user tools to increase the transparency of the and edit its articles; members of this community and the system- evolution of articles and equip editors with more elaborated quality atic behavioral patterns emerging from their interactions form, di- monitors. We also sketch out our evaluation plans and report on rectly or indirectly, intentionally or unintentionally, the decision already accomplished tasks. about which topics are included in the encyclopedia and how they are represented. In order for such “wisdom-of-the-crowds”-driven judgements to have the desired effects, and to propel the creation Categories and Subject Descriptors of high-quality articles - complete, factually correct and unbiased - H.5 [Information Interfaces and Presentation]: Group and Orga- it is essential to study and analyze the underlying community and nization Interfaces—Collaborative computing, Computer-supported its actions, and to devise methods to identify frequently occurring cooperative work, Web-based interaction; H.1 [Models and Prin- social phenomena that are likely to indicate a dysfunctional style ciples]: User/Machine Systems—Human factors of collaboration. Ownership behavior, for instance, wherein an individual editor or a group of editors defend an article from being changed by third parties has been identified in the literature, as we Keywords will see in Section 1.1. Arguably, this behavior has ambivalent Wikipedia, editing behavior, user modeling, collaboration systems, effects: On the one hand, it protects articles from vandalism, on collective intelligence, web science, social dynamics the other hand, it may discourage the introduction of novel information and different points of view by outsiders. There are many 1. PROBLEM STATEMENT other behavioral patterns studied in classical social sciences which can easily emerge in online settings, causing (or facilitating) such In sociology, economy and psychology, there have been decades harmful impairments. Note that by these behavioral patterns we do of research on the emergence of harmful social interaction patterns not refer to purposeful malicious behavior like vandalism or other in certain populations, leading to a vast body of scientific work actions by individual users, but collaborative interaction patterns of aiming to understand, predict, and prevent the occurrence of such conduct in the (well-intended) editing process. phenomena. These include mass hysteria and panics, stock mar- 1 Understanding and systematically detecting social mechanisms ket bubbles and disease spread, to name only a very few. By within Wikipedia is a challenging topic; while empirical studies at- contrast, little is known about how such damaging social dynam- test the existence of a variety of recurring interaction patterns, the ics form in collaborative online environments that potentially bring effects of such interaction often remain unclear, and imprecision and hidden assumptions may lead to unwanted conclusions. This 1A plethora of research work exists on these and many related topics. In the scope of this PhD proposal we cannot introduce them in is partially due to the fact that assessing the quality of Wikipedia their entirety. See [19] (panics), [21] (stock market) and [6] (dis- content is highly contextual; there is not, and will never be, a con- ease spread) as examples. sistent alternative source of information, accepted by all relevant stakeholders, and containing all facts, opinions and viewpoints that Copyright is held by the International World Wide Web Conference should be objectively included in a Wikipedia article. If auto- Committee (IW3C2). IW3C2 reserves the right to provide a hyperlink matic techniques are available, they are naturally restricted to spe- to the author’s site if the Material is used in electronic media. cific sources, which the content of articles is compared against, WWW 2013 Companion, May 13–17, 2013, Rio de Janeiro, Brazil. ACM 978-1-4503-2038-2/13/05. 361 thus identifying only a share of potential inconsistencies or miss- that it is hard for new and occasional editors to change content. ing information. Such methods typically identify major quality As [22] point out, the ratio of reverted edits to the total number flaws in articles, but are less reliable for cases in which such is- of edits has increased with occasional editors experiencing greater sues are slightly less obvious, such as a biased point of view. Aside resistance compared to high-frequency ones.7 Relevant to this sce- from purely content-driven methods, means to detect the emer- nario [13] show that if an editor tries to remove words from an gence of harmful editor behavior patterns promise a complemen- article, the probability of her edit to be reverted increases signifi- tary approach; once such behavior occurs, the system generates cantly with the number of article revisions the removed words had warnings, and the community at large can react and assess the cur- ‘survived’ before (normalized for the number of removed words). rent status of an article apart from simple fact checking, sentiment In short, words which have been in the article for a longer period of analysis and biased wording. time are harder to remove, as more trust is placed in older content. Supplying readers and editors with instruments to make the “so- Likewise, viewpoints added in the early days of Wikipedia could cial evolution” of an article transparent and explicit can be an effec- have a much higher probability of being eventually represented in tive tool to induce awareness of possible incorrectness and bias and the article, supported by the observations of Kittur et al. [14, 15]. counter uncritical perception. As Wikipedia matures, so need the The cause of these phenomena seems to lie in the perceived con- methods to secure its quality. Giving Wikipedians (and any Wiki sensus between the editors, which serves as “social proof” [5] of users) more elaborated tools to see complex problematic patterns the correctness of the information. If a majority has (implicitly) in editor behavior is one step in this direction. Our work aims to assessed a content to be right, it is difficult to change it. An “in- provide such methods, which are accurate regarding the way infor- formation cascade” [3] can occur where all participants place their mation is processed and interpreted, and representative with respect decisions on the preceding editors’ choices instead of their own to the communities of readers and editors they claim to apply to. knowledge. All in all, this valid social heuristic may also hamper The Wikimedia foundation has reached similar conclusions and the replacement of outdated content or the revision of biased con- incorporated them into their strategy to advance the Wikpedia plat- tent towards more balanced opinion expressions, at least for new- form. The research planned in this thesis is carried out in the con- comers or anonymous users. text of the RENDER project2, in collaboration with the German Another observation is that certain editors tend to become over- chapter of the Wikimedia foundation. Results and tools will be protective about article content against changes from other edi- deployed on Wikipedia’s sites and promoted by Wikimedia to the tors. Although explicitly discouraged by Wikipedia,8 strong feel- worldwide community.3 ings of ownership towards an article and protective behavior are not uncommon and “[...] run the risk of deterring new community 1.1 Related work member participation.” [23]. Halfaker et al. [13] infer that “[...] The work presented in the following exemplifies that Wikipedia Wikipedia’s review system suffers from a crucial bias: editors ap- articles are increasingly showing traits that hint at the emergence pear to inappropriately defend their own contributions.” Articles of exactly the harmful behavior patterns we aim to detect. A great guarded in such a way naturally run the risk of being biased as majority of the work mentioned here offers descriptive views on new contributions tend to get accepted only if conforming to the single phenomena the authors observed; as a whole, these studies owner’s taste. Members or subgroups

Load more