Low-effort Crowdsourcing: leveraging peripheral attention for crowd work Abstract Crowdsourcing systems leverage short bursts of focused attention from many contributors to achieve a goal. By requiring people’s full attention, existing crowdsourc- ing systems fail to leverage people’s cognitive surplus in the many settings for which they may be distracted, performing or waiting to perform another task, or barely paying attention. In this paper, we study opportunities for low-effort crowdsourcing that enable people to con- tribute to problem solving in such settings. We dis- cuss the design space for low-effort crowdsourcing, and through a series of prototypes, demonstrate interaction techniques, mechanisms, and emerging principles for enabling low-effort crowdsourcing. Figure 1: With a front-facing camera, the emotive voting in- Introduction terface ‘likes’ an image if you smile while the image is on the screen, and ‘dislikes’ if you frown. Faces are blurred for Crowdsourcing and human computation systems leverage anonymity. the cognitive surplus (Shirky 2010) of large numbers of peo- ple to achieve a goal. Existing systems accomplish this by providing entertainment through games with a purpose (von task. For a person walking to catch a bus, the act of taking Ahn and Dabbish 2008), requiring a microtask for access- out their phone, opening an app, and then performing a sim- ing content (von Ahn et al. 2008), and requesting tasks from ple task will likely already require more effort than people workers and volunteers without requiring long-term com- are willing to give in that situation. Leveraging any cognitive mitment from contributors. By capturing people’s attention surplus people may have in such settings will thus require and leveraging their efforts, systems such as the ESP game, accounting for people’s situational context, and introducing reCAPTCHA, and Mechanical Turk enable useful work to novel interaction techniques and mechanisms that can en- be completed by bringing together episodes of focused at- able effective contributions in spite of it. tention from many individuals. In this paper, we explore opportunities in low-effort While crowdsourcing systems often remove the need for crowdsourcing that enable contributions to crowdsourcing sustained, long-term participation, they still require people’s efforts even in situations when people are distracted, per- full attention while performing a task in the system. As such, forming or waiting to perform another task, or peripher- existing crowdsourcing systems are not designed to leverage ally paying attention. Low-effort crowdsourcing is possible people’s cognitive surplus in the many scenarios for which through a mix of low-granularity tasks, unobtrusive input they may be distracted, performing or waiting to perform methods, and an appropriate setting. We introduce interac- another task, or barely paying attention. For example, pos- tion techniques and mechanisms that enable people to do sible scenarios arise when people are on the go (e.g., walk- useful work with lower fidelity input and output, and mech- ing to the bus), waiting (e.g., for a page to load in a mo- anisms for inserting low-effort crowd work into people’s ex- bile browser; for a train to arrive), or performing a boring or isting situational context. mindless task (e.g., calling customer service; watching TV). We take particular advantage of low-effort, incidental, and Such scenarios create situational impairments (Sears et al. peripheral forms of contribution that arise naturally or can 2003) and otherwise demand people’s attention in ways that otherwise be embedded within an existing interaction. As limit people’s ability and interest in performing an auxiliary one example, consider a display of potentially funny images Copyright c 2014, Association for the Advancement of Artificial scrolling across a screen (see Figure 1). At a glance, a person Intelligence (www.aaai.org). All rights reserved. may be able to tell if a particular image is catchy or funny, and their natural reaction (e.g., turning their head; laughing) the web was previously described as incidental crowdsourc- can be tracked with a camera to serve as a signal about the ef- ing (Organisciak 2013). Incidental crowdsourcing formal- fect the image may have on people. This ‘task’ of looking at ized these types of contributions as fundamentally unob- scrolling images at a glance can be performed while people trusive, meaning they exist in the periphery of other tasks, walk by such a public display, or, on their personal displays and non-critical, both for the user in completing them and while they are on the phone or are otherwise distracted. With the system in relying on them. It was also noted that the appropriate design, such tasks may not only result in useful nature of incidental crowdsourcing contributions is gener- data for immediate use or for machine learning, but can also ally descriptive of existing information objects, tends toward enrich people’s lives by filling in dull moments and relieving low-granularity contributions, and favors choices rather than boredom, during and in between tasks and objectives. statements. Our paper differs from this earlier work in ex- This paper makes the following contributions: (1) the de- ploring low-effort crowdsourcing in broader situational, in- sign space for low-effort crowdsourcing, drawing from lit- teractive, and technological contexts. erature, existing systems, and early prototyping experience; Unobtrusiveness is an important consideration because (2) a series of prototype systems that we built to demonstrate even lightweight feedback mechanisms can be intrusive if what’s possible with low-effort crowdsourcing systems; and they disrupt the user from their primary task. As one ex- (3) interaction techniques, mechanisms, and emerging prin- ample, alerts within mobile applications for a user to “rate ciples for low-effort crowdsourcing that can apply across my app” may disrupt the user experience and annoy the problem domains and user scenarios. We focus in this paper user (Friedman 2013). In designing low-effort crowdsourc- on demonstrating novel techniques and discussing the op- ing interactions, we face a similar challenge in that we aim portunities and challenges in low-effort crowdsourcing, and to leverage people’s efforts while they are involved with an- not on empirical studies to validate any particular prototype. other task. But in addition, the task we wish them to perform That said, in a later section we discuss lessons learned from may or may not be related to their primary task, which re- our prototyping experience, which will inform the design of quires additional consideration. future low-effort crowdsourcing systems. A few recent works explore opportunities for crowdsourc- ing on-the-go by replacing traditional lock screens on mo- Related Work bile phones with tasks that can be completed in a few sec- A popular and effective approach for promoting participa- onds. For example, Twitch (Vaish et al. 2014) replaces the tion in crowdsourcing efforts is to embed useful work in ac- lock screen with census, photo ranking, and data verification tivities that people willingly partake. For example, the ESP tasks. Similarly, Slide to X (Truong, Shihipar, and Wigdor game (von Ahn and Dabbish 2008) collects labels for im- 2014) replaces the lock screen with tasks that collect geo- ages as a side effect of people playing an enjoyable game. graphical and personal health information. While these sys- reCAPTCHA (von Ahn et al. 2008) digitizes books as a side tems share some of the same design challenges as our work effect of people gaining access to valuable content by ver- in that tasks need to be doable in short durations of time and ifying that they are human. Duolingo translates content on have low demands on cognitive load, their requirements dif- the web as a side effect of people learning a new language. fer in that such applications can interrupt the user from their With low-effort crowdsourcing, we extend this approach by primary task briefly (in this case, unlocking and using their designing tasks and interactions that people partake in will- phone). In contrast, our work on low-effort crowdsourcing ingly despite their situational impairment, while also gener- seeks to enable contributions within the natural flow of in- ating useful data as a side effect. teraction, with the goal of collecting useful data and enrich- Perhaps the lowest level of effort required for partici- ing people’s existing interactions without disruption. Low- pation in crowdsourcing arise through opportunistic com- effort crowdsourcing systems can thus promote contribu- munitysensing (Lane et al. 2008), in which applications tions through the duration of another task, and apply to a collect data passively as participants go about their day. non-overlapping set of scenarios. For instance, communitysensing applications collect data from smartphones to detect and predict urban mobility pat- Design Space terns (Vent 2013), noise pollution (Kanjo 2010), and traf- We identify a number of design opportunities and challenges fic conditions (Matthews 2013). While useful for specific for enabling crowd work by people who may be distracted, applications that garner widespread participation, passively performing or waiting to perform another task, or periph- collecting data is limited by the regularity of participants’ erally paying attention. This section explores the design mobility patterns (Gonzalez, Hidalgo, and Barabasi