Community, Cooperation, and Conflict in Wikipedia

John Riedl

GroupLens Research University of Minnesota

UNIVERSITY OF MINNESOTA Wikipedia & the “Editor Decline”

Number of Active Editors

UC Irvine 2012 2 Vision

Community Health Humans + Computers Synthetic vs. Analytic Research

UC Irvine 2012 3 Readership vs. Creation Date

UC Irvine 2012 4 Search Engine Test vs Creation Date

UC Irvine 2012 5 UC Irvine 2012 6 Community Health

We can tell when a community has failed. Our goal is to know before it fails, and to know what to do to prevent failure.

UC Irvine 2012 7 UC Irvine 2012 8 Reader Contributor Collaborator Leader

UC Irvine 2012 9 Creating, Destroying, and Restoring Value in Wikipedia (Group 2007) Reid Priedhorsky Jilin Chen Shyong (Tony) K. Lam Katherine Panciera Loren Terveen John Riedl

UNIVERSITY OF MINNESOTA Probability of View Damaged

UNIVERSITY OF MINNESOTA WP:Clubhouse? An Exploration of Wikipedia’s Gender Imbalance

WikiSym 2011 Shyong (Tony) K. Lam University of Minnesota Anuradha Uduwage University of Minnesota Zhenhua Dong Nankai University Shilad Sen Macalester College David R. Musicant Carleton College Loren Terveen University of Minnesota John Riedl University of Minnesota UNIVERSITY OF MINNESOTA Gender Gap – Overall

2009 editor cohort – females are… 16% of known-gender editors [1 in 6] 9% of known-gender edits [1 in 11]

UC Irvine 2012 13 H1a: Gap-Exists Gender Gap – Historical

Is Wikipedia’s gender gap closing?

Many tech gender gaps are narrowing… Internet, Smartphones, online games

…and some have reversed… Blogging, Facebook, Twitter

UC Irvine 2012 15 Gender Gap Over Time

UC Irvine 2012 16 Movie Gender – Article Length

Male Female Audience Audience Wikimedia Foundation – Vision

“Imagine a world in which every single human being can freely share in the sum of all knowledge. That’s our commitment.”

UC Irvine 2012 18 Collier & Bear, CSCW 2012

H1: Female editors don’t like conflict. SUPPORTED H2: Female editors have lower confidence. SUPPORTED H3: Female contributors prefer to share and collaborate. SUPPORTED H4: Female contributors have less time? REVERSED

UC Irvine 2012 19 Reader Contributor Collaborator Leader

UC Irvine 2012 20 SuggestBot

Cosley et al., IUI 2007

UNIVERSITY OF MINNESOTA How do people find tasks?

Randomly Chronologically Community needs Alphabetically

UC Irvine 2012 22 UC Irvine 2012 23 SuggestBot Results

Intelligent recommendations 4x more likely to be acted on Type of intelligence not as important Live on Wikipedia; Join at: o http://en.wikipedia.org/wiki/User:SuggestBot

SuggestBot 2.0: Balance interest with value

UC Irvine 2012 24 Articles you might like to edit, from SuggestBot

SuggestBot predicts that you will enjoy editing some of these articles. Have fun!

Traditional Readership Quality

Both dimensions Combined

UC Irvine 2012 25 Vision

Community Health Humans + Computers Synthetic vs. Analytic Research

UC Irvine 2012 26 UC Irvine 2012 27 UC Irvine 2012 28 Reader Contributor Collaborator Leader

UC Irvine 2012 29 He Says, She Says: Conflict and Coordination in Wikipedia

Aniket Kittur, Bongwon Suh, Bryan Pendleton, Ed Chi

CHI 2008

UNIVERSITY OF MINNESOTA Types of work

Article Talk, user, procedure

Direct work Indirect work Immediately Coordination, conflict consumable

Maintenance work Reverts, vandalism

UC Irvine 2012 31 Machine Learning: Predicting Conflict

UC Irvine 2012 32 The Effects of Diversity on Group Productivity and Member Withdrawal in Online Volunteer Groups

Jilin Chen Yuqing Ren John Riedl

CHI 2010

UNIVERSITY OF MINNESOTA “A diverse collection of independently- deciding individuals is likely to make certain types of decisions and predictions better than individuals or even experts.”

UNIVERSITY OF MINNESOTA However, difference in opinions and backgrounds can also cause conflicts, protests and wars sometimes …

UNIVERSITY OF MINNESOTA Newcomers vs. Old-timers “This user is sick of the bullshit on wikipedia. It is my belief that wikipedia … is increasingly becoming a place for a small group that is in power to force their views upon the world.”

UNIVERSITY OF MINNESOTA What is Diversity?

Difference among individuals on: o Demographic attributes: age, gender, race o Informational attributes: tenure, education level, knowledge and interest areas o Value attributes: personality, belief

UC Irvine 2012 37 What is Diversity?

Difference among individuals on: o Demographic attributes: age, gender, race o Informational attributes: tenure, education level, knowledge and interest areas o Value attributes: personality, belief

UC Irvine 2012 38 UC Irvine 2012 39 UC Irvine 2012 40 Analysis Approach

Hierarchical Linear Model o Project quarters within the same project are correlated Incremental Approach o Start with a null model, and increase its complexity incrementally o Ensure that the predictors in the model are useful

UC Irvine 2012 41 Tenure Diversity : Results

UC Irvine 2012 42 Interest Diversity : Results

UC Irvine 2012 43 Future Research

Direct Manipulation of Group Diversity o Is it more beneficial to introduce a newcomer or an old-timer to a particular project? Examination of Group Processes o Do members yell at each other more often when the group is too diverse?

UC Irvine 2012 44 Don't bite the newbies: How Reverts affect the Quantity and Quality of Wikipedia Work

Aaron Halfaker, Niki Kittur and John Riedl WikiSym 2011

UC Irvine 2012 45 Wiki solution: The revert

 The revert

UC Irvine 2012 46 Research Question:

What effect do reverts have on editor activity?

UC Irvine 2012 47 Measuring Changes: Activity Deltas

UC Irvine 2012 48 Measuring Changes: Activity Deltas

UC Irvine 2012 49 Measuring Changes: Activity Deltas

UC Irvine 2012 50 Effect on Quality of work

UC Irvine 2012 51 Summary

 Being reverted predicts:

 Decreases in quantity of work

 Increase in quality of work

 Overall increase in productivity!

 Curmudgeons

 Being reverted by an old-timer intensifies the decrease in quantity, no change to quality

 Highly experienced editors should be kept away from newbies.

UC Irvine 2012 52 Revert percent over time, by PARCeditor Social Cognition class Blog, 2009 Percentedits reverted

Edits per month

UC Irvine 2012 53 Revert Rate vs. Tenure 9% FemaleRevert newcomers get Chart reverted 8% more often than male newcomers Female

7% Male

6% Lam et al. WikiSym 2011: WP:Clubhouse 5%

2% Reverted Edit Percentage 1%

0% 1 2-3 4-7 8-15 16-31 32-63 64-127 128-255 Stage of Editor Tenure (Edit #) UC Irvine 2012 54 NICE: Social Translucence through UI Intervention Aaron Halfaker, Bryan Song, D. Alex Stuart, Niki Kittur, John Riedl WikiSym 2011

UC Irvine 2012 55 NICE

 User Interface mod for English Wikipedia

 Installed by the editor performing the revert

 Helps an editor revert gently

 Live and available in English Wikipedia

 http://en.wikipedia.org/wiki/User:EpochFail/NICE

UC Irvine 2012 56 Reverting Normally

UC Irvine 2012 57 Reverting NICE-ly

UC Irvine 2012 58 Field Experiment

 Broadcast to forums & mailing lists used by Wikipedians

 49 reverting editors - Message Box - Newbie Warning - Message Box + Newbie Warning - Control

 487 reverted editors

UC Irvine 2012 59 Results

 Message Box (easy communication)

 Newbies: (-) Quality (Sigh!)

 Non-newbies:

 (+) Project pages, (+) Communication (+) Quality

 Newbie Warning (social awareness)

 Newbies: (+) Article work

 Non-newbies: (+) Quality

UC Irvine 2012 60 Vision

Community Health Humans + Computers Synthetic vs. Analytic Research

UC Irvine 2012 61 Reader Contributor Collaborator Leader

UC Irvine 2012 62 Mortality Rate

Article creation restricted to registered users

Jimmy Wales’ Wikimania plenary: focus on quality instead of quantity

UC Irvine 2012 63 Female Leaders

Females who become Wikipedian are much more likely to become admins

UC Irvine 2012 64 Reporters in Captivity “A journalist is someone who is on no side and a friend of everybody.” Rohde captured by Taliban, Nov 2008 Times gets 7 months news embargo Wales says “mainstream media did not report”, so Wikipedia could embargo Christian Science Monitor not mentioned – though Rohde won Pulitzer there Rohde escapes, June 2009

UC Irvine 2012 65 The Effects of Group Composition on Decision Quality in a Social Production Community Shyong (Tony) K. Lam Jawed Karim John Riedl

University of Minnesota Stanford University

GROUP 2010

UNIVERSITY OF MINNESOTA Wikipedia Article Deletions

About 30% of created articles are deleted

Most deletions are uncontroversial

Disagreements resolved via decision process Articles for Deletion (AfD)

UC Irvine 2012 67 Sample AfD AfD Result (Keep or Delete) AfD NominationLooks like a vote, but it isn’t one Wikipedians call these “!votes”

Discussion and debate

UC Irvine 2012 68 Measuring Decision Quality

Computing a believable quality metric is hard Lots of Wikipedia policies Who are we to judge?

Use implicit community feedback Persistent decision = Good Reversed decision = Bad

About 4% of decisions are reversed in 1 year

UC Irvine 2012 69 Consensus Call Proﬁles Consensus100 Call Profiles - WPWide 90

20 % AfDs Closed as Delete 10

0 0 10 20 30 40 50 60 70 80 90 100 Delete !Vote % UC Irvine 2012 70 Consensus Call Proﬁles 100 Consensus Call Profiles - Keep 90 Wikipedia-Wide 80

70 Keep-Biased Admin

20 % AfDs Closed as Delete 10

0 0 10 20 30 40 50 60 70 80 90 100 Delete !Vote % UC Irvine 2012 71 Consensus Call Proﬁles 100 Consensus Call Profiles - All 90 H4: Biased-Admin-Worse Delete-Biased Admin 80 Supported Keep-biasedWikipedia-Wide admins: 70 +23% oddsKeep-Biased Admin of bad keep decision 60 –8% odds of bad delete decision

20 % AfDs Closed as Delete 10

0 0 10 20 30 40 50 60 70 80 90 100 Delete !Vote % UC Irvine 2012 72 Result Summary

Factor Effect on Decision Quality Size Bigger  better, diminishing returns Formation None in most cases, despite apparent recruitment bias Newcomers Newcomer participation  worse Tenure diversity High diversity  better Biased admin Decision with bias  worse Decision against bias  better

Diverse Knowledge Independent Fair

UC Irvine 2012 73 Future Directions

Build automated tools to detect problems or opportunities for improvement

Explore Wikipedia’s possible biases (recruitment, administrative)

Study AfDs qualitatively to understand why

UC Irvine 2012 74 Vision

Community Health Humans + Computers Synthetic vs. Analytic Research

UC Irvine 2012 75 Thank you!

Acknowledgements GroupLens Research Wikimedia Foundation National Science Foundation o grants: IIS 05-34420, 07-29286, 08-08711, 0968484, and OCI-0943148

John Riedl [email protected]

UC Irvine 2012 76 Community, Cooperation, and Conflict in Wikipedia

John Riedl

GroupLens Research University of Minnesota

UNIVERSITY OF MINNESOTA Bit Bucket

UC Irvine 2012 78