Moods: Building Massive Open Online Diaries for Researchers, Teachers and Contributors
Total Page:16
File Type:pdf, Size:1020Kb
MOODs: Building Massive Open Online Diaries for Researchers, Teachers and Contributors Sandy J. J. Gould Sarah Wiseman Abstract UCL Interaction Centre UCL Interaction Centre Internet-based research conducted in partnership with University College London University College London paid crowdworkers and volunteer citizen scientists is an London, WC1E 6BT London, WC1E 6BT increasingly common method for collecting data from [email protected] [email protected] large, diverse populations. We wanted to leverage web- based citizen science to gain insights into phenomena Dominic Furniss Ioanna Iacovides that are part of people’s everyday lives. To do this, we UCL Interaction Centre UCL Interaction Centre developed the concept of a Massive Open Online Diary University College London University College London (MOOD). A MOOD is a tool for capturing, storing and London, WC1E 6BT London, WC1E 6BT presenting short updates from multiple contributors on [email protected] [email protected] a particular topic. These updates are aggregated into public corpora that can be viewed, analysed and Charlene I. Jennett Anna L. Cox shared. MOODs offer a novel method for crowdsourcing UCL Interaction Centre UCL Interaction Centre diary-like data in a way that provides value for University College London University College London researchers, teachers and contributors. MOODs also London, WC1E 6BT London, WC1E 6BT come with unique community-building and ethical [email protected] [email protected] challenges. We describe the benefits and challenges of MOODs in relation to Errordiary.org, a MOOD we Permission to make digital or hard copies of part or all of this work for personal created to aid our exploration of human error. or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of Author Keywords this work must be honored. For all other uses, contact the owner/author(s). Citizen science; crowdsourcing; human error; ethics; Copyright is held by the owner/author(s). diary studies; Twitter CHI 2014, Apr 26 - May 01 2014, Toronto, ON, Canada ACM 978-1-4503-2474-8/14/04. http://dx.doi.org/10.1145/2559206.2581140 ACM Classification Keywords H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. Introduction Massive Open Online Diaries Diary studies have been a popular method for Crowdsourcing, whether through paid, work-centric investigating HCI problems ‘in the wild’. For example, platforms such as Amazon’s Mechanical Turk or through diaries have been used to investigate topics such as research-driven citizen science projects has proved to user information requirements [10] and task be a useful method for collecting large quantities of management strategies [4]. Although pen-and-paper data. Previous research efforts have used citizen approaches to diary studies have been augmented and science techniques – where volunteers collaborate with enriched by technology-supported techniques for professional scientists to conduct scientific research – recording experiences [e.g., 10], the relationship to investigate questions in a number fields (e.g., between researchers, diary keepers and entries has astronomy [7]). Sometimes crowdsourcing approaches remained largely unchanged; participants submit their have simply replicated standard methods online. Other entries to researchers and the data are presented research has used crowdsourcing to pioneer entirely anonymously in aggregate. new techniques of data collection. A MOOD is a hybrid method for data collection that combines traditional Errordiary entry #1903: In most cases, unidirectional relationships where diary techniques with the benefits of an open online “Don’t enter your PIN instead entries flow from participants to researchers are collaborative space. of the tip when paying for sufficient to meet the data collection requirements of a coffee. Makes for a very particular research question. However, such an A MOOD invites individual contributors to make short expensive treat £XXXX.” approach does not leverage the potential for diaries to diary-like posts on a particular topic of interest to both be community-built, collaborative spaces that produce researchers and contributors. Contributions can be value for researchers, teachers and contributors. made through microblogging services, like Twitter, or through more traditional website-based interactions. This paper presents a model for diary-style data These short, timely updates are then added to a collection that harnesses an open, accessible, web- publicly accessible corpus from which they can be based platform to capture, display and curate diary searched, shared or analysed. entries. We call this model of participation a Massive Open Online Diary. We explore the potential benefits Aggregating short posts into online corpora is an idea and challenges associated with MOODs through a long- that has been developed in the past by a few online running exemplar, the Errordiary project. In particular, communities. For instance, the journalist-run Everyday we focus on the contribution that MOODs can make to Sexism project (everydaysexism.com) exists to collect research, teaching and participants while considering examples of casual sexist behaviour experienced by the community-building and ethical challenges that a contributors. Posts are made directly to the site or project of this type faces. Of course, there are other harvested from Twitter. The project is a good example outcomes and challenges that MOODs must address, of online activism – a cause has been identified and the but we chose these benefits and challenges as we see project serves to highlight the need for change. User them as essential considerations for any MOOD. contributions reinforce the perception that action is required while providing illustrative examples that as part of the project, the solutions we developed and make it easy for visitors to grasp the issue. the broad applicability of these problems to potential MOOD implementations in future. Despite the success of these kinds of projects in gaining large user bases and significant media attention [1], we Research are unaware of any projects that have used a MOOD- The primary objective of MOODs should be to collect like approach to meet the goals of a research agenda. useful research data; this distinguishes them from In this paper we present such a project, through a case projects like Everyday Sexism. As MOODs are living study of a long-running investigation of human error, corpora without defined finishing dates, data can be the Errordiary project. drawn-off as required to answer questions that might evolve over the course of a project. Data from The Errordiary project Errordiary has already been employed in novel The Errordiary project (errordiary.org) has been research. For instance, data on the individual strategies running since 2009. The goal of the project is to collect contributors have used to avoid error were collated and examples of everyday human errors that are valuable coded to determine whether such strategies can be to researchers, teachers and contributors. Errordiary grouped by a smaller set of types [6]. builds on a tradition of diary-led studies of human error, for instance, Sellen used diary studies to develop As well as the raw data they produce, MOODs can also a taxonomy of everyday errors [9]. offer researchers meta-insights into their research Contributors can add their entries agendas. For example, by calling for contributions on through Twitter using the We took a diary-like approach for the project because the broad topic of everyday errors, we have come to #errordiary hashtag or directly diary entries incur relatively low time costs for better understand the types of events that people see through the Errordiary website. contributors and researchers and are suitable for as errors. The ‘favouriting’ system built-in to Errordiary longitudinal use. In an effort to encourage pithy allows contributors to select the entries that they think contributions, reduce the time costs involved in best represent the project. Such tools allow for a participating and to reduce the effort required for participatory form of research that provides researchers people browsing the corpus to read the entries, we with perspective on their research questions that they initially limited the platform for making contributions to might otherwise lack. Twitter. The length of entries was thus limited to 140 characters. Errordiary currently has over 2000 entries Teaching from over 130 contributors and covers everything from The openness of MOODs also makes them an ideal errors during beverage-making to lost keys. resource for use in teaching. In the case of Errordiary, we have used the corpus extensively in our teaching of The general concept of a MOOD was developed from human error to undergraduate and postgraduate our experience with Errordiary. In the rest of this paper students [11]. We have found the entries particularly we describe some of the problems that we had to solve useful for classification exercises; asking students to Errordiary entry #1264: “Got the coffee strength and coffee amount knobs mixed up. Got an over-flowing mug of weak coffee.” classify a random selection of entries gives them insight number of techniques for building engagement with the into the strengths and weaknesses of taxonomies and project.