Online Citizen Science Projects: an Exploration of Motivation, Contribution and Participation

Home , Circumbinary planet, Planet Hunters, World Community Grid

Open Research Online The Open University’s repository of research publications and other research outputs

Online citizen science projects: an exploration of motivation, contribution and participation

Thesis

How to cite:

Curtis, Vickie (2015). Online citizen science projects: an exploration of motivation, contribution and participation. PhD thesis The Open University.

For guidance on citations see FAQs.

c 2015 The Author

Version: Version of Record

Link(s) to article on publisher’s website: http://dx.doi.org/doi:10.21954/ou.ro.0000a4ﬀ

Copyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyright owners. For more information on Open Research Online’s data policy on reuse of materials please consult the policies page. oro.open.ac.uk

Online citizen science projects: an exploration of motivation, contribution and participation

Vickie Curtis

MA (Hons) Biological Anthropology MPhil Human Nutrition MSc Science and Society

Submitted for the degree of Doctor of Philosophy Institute of Educational Technology The Open University October 2014

Abstract The number of online citizen science projects has increased significantly over the past decade, yet some aspects of participation are poorly understood as is the motivation behind the involvement of the scientists who set up these projects, and the citizen scientists who take part.

This thesis explores three different online citizen science projects (Foldit, Folding@home and

Planet Hunters) using a case study approach and data collected through online surveys, interviews and participant observation. It explores the motivations that initiate and sustain participation, and it examines the various ways individuals can contribute to these projects. It also investigates how participants (both professional scientists and citizen scientists) interact online. A number of theoretical models of motivation and participation are considered.

While many individuals register to take part in these projects, only a small proportion become active participants. These active citizen scientist volunteers are motivated to participate because they want to make a contribution to science or have a background interest in science. Scientists set up these projects in order to get help analysing large volumes of data, particularly those that require human pattern recognition or problem-solving skills. The complexity of the project task and the presence of certain technological features can affect how participants interact with each other, and how they contribute. Tasks that are complex are more likely to present opportunities for cooperation and collaboration, and may foster the development of online communities of practice.

The findings of this research suggest that online citizen science projects have been important in making scientific research more open for a number of distributed volunteers. These individuals have responded to the challenges presented by these projects, increasing their scientific and technical understanding, and self-organising into various roles and teams in order to produce new knowledge.

Acknowledgments

I would like to thank my three supervisors Dr Richard Holliman, Professor Eileen Scanlon and Dr Ann Jones for their support and advice over the past three years, especially during the writing of this thesis. I would also like to thank Dr Trevor Collins for his feedback on earlier drafts of this work.

This research would not have been possible without access to the three online citizen science projects that have been the focus of my study. I would like to thank Professor

David Baker, Professor Zoran Popovich and Dr Seth Cooper of Foldit; Professor Vijay

Pande of Folding@home; and Dr Chris Lintott of the Zooniverse and Planet Hunters. I would also like to thank Jordan Raddick of the Zooniverse, for providing me with data relating to participant activity on Planet Hunters. I am also indebted to the many participants, scientists and developers who kindly took the time to answer my questions.

Finally, but by no means least, I would like to thank my family for their support and patience during the past three years, particularly my husband David, and my children,

Rory and Dotty. I am also incredibly grateful to my mother, Liz, and my parents-in-law

Marjory and Peter Craig who have always been there to help out with child care during school holidays, and most importantly, during the final phase of writing this thesis.

Contents

Online citizen science projects: an exploration of motivation, contribution and participation ......

Abstract ...... i

Acknowledgments ...... ii

Index of Tables ...... x

Index of Figures ...... xi

Chapter 1: Introduction...... 1

1.1 Overview and development of online citizen science ...... 1

1.2 The impact of digital and communication technologies on citizen science ...... 3

1.3 The rise of online citizen science ...... 5

1.4 Gaps in our understanding of online citizen science projects ...... 8

1.5 Organisation of the thesis ...... 12

Chapter 2: Review of the Literature ...... 15

2.1 Introduction ...... 15

2.2 Main features of online citizen science ...... 16

2.3 Who takes part in online citizen science? ...... 18

2.4 Motivation to participate in online citizen science projects ...... 22

2.4.1 Motivation and volunteerism ...... 22

2.4.2 Motivation and other types of activity ...... 24

2.4.3 Motivation and commons-based peer production ...... 26

iii

2.4.4 Motivation and citizen science ...... 28

2.4.5 Motivation and online citizen science ...... 30

2.5 Interaction in online citizen science projects ...... 40

2.6 Contribution to online citizen science projects ...... 43

2.7 Conclusion ...... 49

Chapter 3: Methodology and Methods ...... 51

3.1 Methodological approach ...... 51

3.1.1 A mixed methods approach ...... 51

3.1.2 Case studies as a mixed-methods approach ...... 55

3.1.3 Online citizen science projects as case studies ...... 58

3.2 The online research setting and ethical considerations ...... 60

3.3 Methods ...... 62

3.3.1 The selection of the cases ...... 63

3.3.2 Streams of evidence to be considered...... 65

3.4.1 Online survey data ...... 82

3.4.2 Interview data ...... 84

3.4.3 Participant-observer notes ...... 86

3.5 Conclusion ...... 87

Chapter 4: Description of the projects ...... 89

4.1 Foldit (www.fold.it) ...... 89

4.1.1 Underlying science and project task ...... 91

4.1.2 Foldit players and teams ...... 94

4.1.3 Active playing population ...... 98

4.1.4 Interaction between Foldit participants ...... 99

4.1.5 Participant experience ...... 102

4.1.6 Foldit: summary ...... 109

4.2 Folding@home (http://folding.stanford.edu) ...... 109

4.2.1 Underlying science and project task ...... 110

4.2.2 Participating in Folding@home ...... 112

4.2.3 Hardware enthusiasts and ‘overclockers’ ...... 115

4.2.4 Participant numbers ...... 117

4.2.5 Opportunities for interaction ...... 118

4.2.6 Participant experience ...... 120

4.2.7 Folding@home summary ...... 127

4.3 Planet Hunters (www.planethunters.org) ...... 128

4.3.1 Underlying science and the project task ...... 128

4.3.2 Active playing population ...... 131

4.3.3 Interaction between participants ...... 134

4.3.4 Participant experience ...... 135

4.3.5 Planet Hunters summary ...... 140

4.4 Conclusion ...... 140

Chapter 5: Foldit results ...... 143

5.1 The online survey results ...... 143

5.1.1 Demographic characteristics ...... 143

5.1.2 Background interest in science and participation in related activities ...... 145

5.1.3 Patterns of participation ...... 147

5.1.4 Motivation and reward ...... 150

5.1.5 Interaction with other participants ...... 154

5.2 Interview feedback ...... 155

5.2.1 Interviews with Foldit players ...... 156

5.2.2 Interviews with members of the Foldit team ...... 169

5.3 Summary of results ...... 178

Chapter 6: Folding@home results ...... 181

6.1 The online survey results ...... 181

6.1.1 Demographic characteristics ...... 182

6.1.2 Background interest in science and participation in related activities ...... 184

6.1.3 Patterns of participation ...... 185

6.1.4 Motivation and reward ...... 188

6.1.5 Interaction with other participants ...... 192

6.1.6 Perceived contribution to Folding@home...... 193

6.1.7 Further quantitative analysis ...... 195

6.2 Interview feedback ...... 195

6.2.1 Interviews with citizen scientists ...... 196

6.2.2 Interviews with project team members ...... 210

6.3 Summary of Results...... 216

Chapter 7: Planet Hunters Results ...... 219

7.1 Online survey results ...... 219

7.1.1 Demographic characteristics ...... 219

7.1.2 Background interest in science ...... 222

7.1.3 Patterns of participation ...... 223

7.1.4 Motivation and reward ...... 225

7.1.5 Interaction and contribution ...... 228

7.2 Interview feedback ...... 231

7.2.1 Interviews with citizen scientists ...... 231

7.2.2 Interviews with the project team ...... 243

7.3 Summary of results ...... 253

Chapter 8: Discussion and comparison of results ...... 256

8.1 Who takes part in online citizen science projects? (RQ1) ...... 256

8.1.1. The predominance of male respondents ...... 259

8.1.2 Level of education and general interest in science ...... 261

8.1.3 The appeal of online citizen science projects ...... 264

8.2 What motivations initiate and sustain participation in online citizen science

projects? (RQ2) ...... 266

8.2.1 Initiating participation ...... 266

vii

8.2.2 Sustaining participation ...... 270

8.2.3 Models of motivation ...... 270

8.2.4 Motivations of project scientists and developers ...... 275

8.3 How do motivations vary between different types of online citizen science projects

and their associated tasks? (RQ 3) ...... 278

8.4 How and why do project participants interact online? (RQ4) ...... 282

8.4.1 Interaction between citizen scientists ...... 282

8.4.2 Interaction between project team members and citizen scientists ...... 285

8.5 How can contribution to online citizen science projects be characterised? (RQ5) .291

8.5.1 Contribution of citizen scientists ...... 291

8.5.2 Patterns of participation and contribution ...... 292

8.5.3 Contribution of project team members ...... 298

8.6 How do participants perceive their role in the project? (RQ 6) ...... 299

8.7 Conclusions ...... 302

Chapter 9: Conclusion ...... 305

9.1 Contributions of this work ...... 306

9.1.1 Contributions to knowledge ...... 306

9.1.2 The inter-relationship between contribution, motivation and interaction ...... 312

9.1.3 Methodological insights ...... 314

9.1.4 Revisiting the typology ...... 316

viii

9.1.5 Considerations for scientists and developers interested in online citizen

science ...... 318

9.2 Limitations of this study ...... 321

9.3 Future work ...... 322

9.4 Final reflections ...... 324

References ...... 326

Appendix A: Cover page for Foldit online survey ...... 361

Appendix B: Foldit survey...... 362

Appendix C: Folding@home survey ...... 368

Appendix D: Planet Hunters Survey ...... 375

Appendix E Interview questions for citizen scientists...... 381

Interview Questions for Foldit Players ...... 381

Interview questions for Planet Hunters participants ...... 382

Interview questions for Folding@home participants (beta team and overclockers) ....383

Appendix F: Interview questions for scientists and developers ...... 384

Interview questions for Foldit Team ...... 384

Interview Questions for PH Scientific Team ...... 385

Interview Questions for Folding@Home Project Team ...... 386

Index of Tables 1.1 Proposed typology of online citizen science projects……………………………………….. 10 2.1 Demographic data of citizen scientists …………………………………………………………….. 19 2.2 Studies exploring motivation in online citizen projects………………...... 31 2.3 Main features of lightweight and heavyweight peer production………………………. 48 3.1 Selection criteria for cases……………………………………………………………………………….. 63 3.2 Projects considered as case studies………………………………………………………………….. 64 3.3 Projects selected for case studies…………………………………………………………………….. 65 3.4 Timing of data collection and generation…………………………………………………………. 66 3.5 The continuum of participation………………………………………………………………………… 69 3.6 Participant Observer Protocol………………………………………………………………………….. 70 3.7 How participant surveys were advertised and final numbers of respondents…………………………………………………………………………………………………………… 74 3.8 Details of the interviews carried out………………………………………………………………… 80 4.1: Player roles within Foldit…………………………………………………………………………………. 108 5.1 Occupation or profession of respondents………………………………………………………… 145 5.2 Emerging themes from interviews with Foldit players……………………………………… 156 5.3: Emerging themes from interviews with Foldit developers………………………………. 170 6.1 Occupation / profession of respondents………………………………………………………….. 183 6.2 Emerging themes from interviews with Folding@home participants……………….. 196 6.3 Emerging themes from Beta Testers interviews……………………………………………….. 204 6.4 Emerging themes from overclocker interviews………………………………………………… 207 6.5 Emerging themes from interviews with Folding@home project team……………… 211 7.1 Profession of respondents……………………………………………………………………………….. 221 7.2 Emerging themes from interviews with citizen scientists…………………………………. 231 7.3 Emerging themes from interviews with project scientists………………………………… 244 8.1 Summary of main motivations that initiate and sustain participation for citizen scientists………………………………………………………………………………………………………………… 266 8.2 Motivational framework…………………………………………………………………………………… 274 8.3 Additional project tasks and roles…………………………………………………………………….. 292 9.1 Revised typology – online citizen science project tasks…………………………………….. 317 9.2 Future research projects…………………………………………………………………………………… 324

Index of Figures 1.1 Timeline of research activities…………………………………………………………………. 14 2.1 ‘Reader-to-leader’ Framework…………………………………………………………………. 46 3.1 Descriptors of case studies………………………………………………………………………. 57 3.2 Mapping out the design for the online citizen science case studies…………. 59 3.3 Data collection process (‘exploratory sequential’)…………………………………… 67 3.4 Data Analysis (‘convergent concurrent design)………………………………………… 81 4.1 Foldit publication in Nature Structural and Molecular Biology…………………. 90 4.2 Third Foldit intro puzzle……………………………………………………………………………. 92 4.3: Foldit Science Puzzle……………………………………………………………………………….. 93 4.4: Foldit player profile page………………………………………………………………………… 95 4.5 Example of team profile…………………………………………………………………………… 97 4.6 Example of Foldit forum discussion………………………………………………………….. 100 4.7 Transcript of developer chat…………………………………………………………………… 107 4.8 Process of protein folding……………………………………………………………………….. 110 4.9 Folding@home progress page…………………………………………………………………. 112 4.10 Folding@home participant profile…………………………………………………………. 113 4.11 Customised ‘folding rig’…………………………………………………………………………. 116 4.12 A selection of discussion threads from Folding@home forum……………….. 119 4.13 General discussion area of Folding@home forum…………………………………. 120 4.14 Sharing achievements on the EVGA folding forum………………………………… 122 4.15 ‘Why we DC’ from the HardOCP forum………………………………………………….. 123 4.16 Feedback relating to changes in BigAdv points scheme from EVGA forum…………………………………………………………………………………………………………….. 125 4.17 Light curve for one of the stars in the Kepler field………………………………….. 129 4.18 Planetary transit…………………………………………………………………………………….. 130 4.19 Pattern of participation in Planet Hunters…………….……………………………….. 133 4.20 Participation ‘ribbon’ for Zooniverse………………………………………………………. 137 5.1 Highest educational level attained…………………………………………………………… 144 5.2 Highest educational qualification in a STEM subject…………………………………. 144 5.3 Participation in other citizen science projects…………………………………………… 146 5.4 Participation in science-related activities during the past year…………………. 147

5.5 Length of time respondents have been playing Foldit………………………………. 148 5.6 Number of hours per week respondents play Foldit…………………………………. 149 5.7 Number of hours per week respondents participate in online discussions.. 150 5.8 Why respondents decided to participate in Foldit……………………………………. 151 5.9 What respondents like best about Foldit………………………………………………….. 152 6.1 Highest educational level attained by respondents………………………………….. 183 6.2 Participation in other citizen science projects………………………………………….. 184 6.3 Other science-related activities undertaken during the last 12 months……. 185 6.4 Duration of participation in Folding@home...... 186 6.5 Participation in online in Folding@Home discussions and forum……………. 187 6.6 Why respondents decided to participate in Folding@home...... 188 6.7 What respondents like best about Folding@Home...... 190 6.8 Do respondents feel as though they are involved in scientific research……. 194 7.1 Highest level of educational attainment…………………………………………………… 220 7.2 Other science-related activities undertaken during the last 12 months……. 222 7.3 Other citizen science projects that respondents have participated in……… 223 7.4 The number of hours respondents spend participating in PH per week……. 224 7.5 Why respondents decided to try Planet Hunters……………………………………… 225 7.6 What respondents liked best about the project……………………………………….. 227 7.7 Do you think you should be rewarded for your participation in Planet Hunters?...... 228 7.8 How would you describe your interaction with other participants?...... 229 8.1 Adaptation of the ‘reader-to-leader’ framework for online citizen science projects………………………………………………………………………………………………………….. 295 8.2 Lightweight and heavyweight scales of peer production and position of the three online citizen science studies investigated in this research…………….. 298 9.1 Inter-relationship between contribution, motivation and interaction………. 312

xii

Chapter 1: Introduction

1.1 Overview and development of online citizen science

Citizen science is a collective term for projects that engage both professional scientists and non-scientists in the process of gathering, evaluating and/or computing various scientific data (Kostadinova 2011). It has been around for over a century, and its development can be linked to the ‘professionalisation’ of science that began in the late nineteenth century, when science began to emerge as a distinct occupation (Vetter 2011,

Miller-Rushing et al., 2012). Citizen science projects are often ‘top-down’ researcher-led initiatives, where professional scientists enlist the help of volunteers to either collect or evaluate data usually after a brief period of training (Marks 2013). In some cases, this data can lead to the production of scientific publications or it may help to inform public policy (Irwin 1995, Stilgoe 2009, Roy et al., 2012). Citizen science projects may also have a ‘bottom-up’ approach in which citizens influence the selection of scientific research topics, especially in the area of environmental activism (Irwin 1995, Ottinger 2010,

Conrad and Hilchey 2011).

In 1995 Irwin used ‘citizen science’ to refer to the involvement of citizens in addressing local environmental issues that relied on the collection and analysis of scientific data

(Irwin 1995). The term was later used and adapted by Bonney and co-workers at the

Cornell Laboratory of Ornithology to describe wider opportunities for non-specialists to become involved in real-world scientific research in a variety of different scientific disciplines (Bonney et al., 2009, Riesch and Potter 2013). These two distinct approaches to citizen science have been described as citizen-led co-created projects with local community groups on the one hand, and scientist-led participation initiatives that are

1 open to all sectors of society on the other (Roy et al., 2012). This research focuses on the latter.

Scientist-led citizen science projects can have multiple aims and purposes and be utilised in a variety of settings, both small and large-scale. A significant proportion of these projects have used volunteers to collect ecological, biological or environmental data

(Silvertown, 2009, Wiggins and Crowston, 2011). This data can be collected from a variety of geographical locations and over time in order to track phenological (seasonal or life-cycle) changes in wildlife, bird migration patterns, or more recently, biological or environmental markers of climate change (Devictor et al., 2010, Howard et al., 2010,

Mayer 2010, Yaukey 2010). Given the geographical and temporal scale, such projects would be difficult, if not impossible, without the contributions of citizen scientists.

As well as fulfilling a specific research aim, citizen science projects can play a role in informal science learning and potentially increase scientific literacy (Cooper et al., 2009,

Kloetzer 2013). Some scientists who set up citizen science projects have used them as opportunities to educate and engage non-specialists (Bonney et al., 2009, Silvertown

2009, Gura 2013). Citizen science projects may also have the potential to produce partnerships between scientists and non-scientists, and introduce non-scientists to the scientific research process (Field and Powell 2001, Bonney et al., 2009).

With the expansion of the Internet and a greater availability of digital tools, there has been a rapid growth in citizen science projects over the past decade or so (Hand 2010,

Wiersma 2010, Gura 2013). Improvements in information and communication technologies (ICT) have made it possible for scientists to manage projects, recruit and communicate with volunteers, collate data, and disseminate research findings more

2 widely (Newman et al., 2011, Könneker and Lugger 2013). These technologies have also made it possible for prospective participants to get involved.

1.2 The impact of digital and communication technologies on citizen science

During the past two decades, developments in ICT have changed the way scientists work in a number of ways (Holliman 2010). This is most notable in the creation and integration of new knowledge between different research areas or disciplines (Scanlon 2013). Digital technologies have also influenced how scientists communicate with one another, and how they communicate with those ‘outside’ the scientific community (Borgman 2007,

Scanlon 2013). For example, online sharing of data has facilitated scientific collaboration; and the rise of ‘open notebooks’, online repositories, and open-access journals has aided the dissemination of scientific results (Grand et al., 2010, Cranshaw and Kittur 2011,

Nielsen 2012).

Scientists are able to communicate more widely with interested non-specialists through websites, blogs, podcasts and through social media (Birch 2010, Kouper 2010, Blank and

Reisdorf 2012). Some maintain that the development of ‘Web 2.0’ technologies which facilitate participatory data sharing and the production of user-generated content, can begin to blur the boundary between professionals and an increasingly informed online public, and that this may have important consequences for the way scientific knowledge is generated (Lievrouw 2010, Stodden 2010, Nielsen 2012).

This rise in digital science and the expansion of new avenues of communication has been referred to as ‘open science’ or ‘Science 2.0’ (Burgelman et al., 2010, Nielsen 2012,

Könneker and Lugger 2013). While these are somewhat ambiguous terms, with

3 conflicting opinions regarding definitions, this phenomenon could be generally described as a trend towards the increased connectivity between scientists, and an increased capability for non-scientists to access science and the scientific community. Open science may allow for greater transparency, as well as greater opportunities for non-specialist participation (Catlin-Groves 2012, Grand et al., 2012, Mansell 2012, Czerniewicz 2013).

The growth and impact of digital technologies, has been accompanied by an increase in the accuracy and productivity of scientific instrumentation and data storage technologies.

This has led to what has become known as the ‘data deluge’, as scientists in some disciplines now acquire, store and mine huge volumes of digital data (Hey and Trefethen

2008, Creighton 2010, Clavin 2013). The increase in data-intensive science has also been referred to as ‘e-science’, and it requires new tools and techniques to organise, filter, share, re-use, recombine and analyse (McFedries 2011, Mayer-Schonberger and Cukier

2013). For example, the Large Hadron Collider generates approximately 15 petabytes1 of data per year when in operation, and the Large Synoptic Survey Telescope which will be in operation in 2022, will produce 100 terabytes2 of data every night (Hey and Trefethen

2008, McFedries 2011).

The developments in ICT seen since the mid-1990’s (in particular, the expansion of the

Internet), along with the ‘data deluge’ has resulted in some important changes in the way scientific research is carried out (Neylon 2011). Furthermore, these developments have also had implications for citizen science and in the type of opportunities available for both scientist and citizen scientist (Wiggins 2010, Kostadinova 2011, Prestopnik and Crowston

2012).

1 A petabyte is one million gigabytes. 2 A terabyte is one thousand gigabytes. 4

By the late 1990’s, some scientists had realised that they would never be able to analyse all of their data on their own, and devised new ways to enlist the help of those outside of their academic institutions (Nov et al., 2010, Schawinski 2011). This has led to what some have referred to as the ‘crowdsourcing’ of science, where interested members of ‘the public’ can help to analyse data produced by instruments that they would not normally have access to (Qadir 2013, Uchoa et al., 2013). Scientists too, now have access (via the

Internet) to many thousands of potential participants in their projects, and are able to accomplish more than was previously thought possible (Hand 2010). As a result of these developments, some citizen science projects are conducted entirely through the Internet and participants help to analyse large sets of data that has been provided by the project scientists. These projects have been referred to as online citizen science (Holliman and

Curtis 2014).

1.3 The rise of online citizen science

One of the first online citizen science projects to emerge from this mixture of abundance of data, and the expansion of the internet was SETI@home3 (Anderson 2004). In 1999, scientists from the University of California, Berkeley, asked members of the general public to volunteer their idle PC processing capacity to analyse data produced by radio telescopes searching for signs of extra-terrestrial intelligence. Hundreds of thousands of people have taken part in SETI@home4, and its software platform has been adapted for use in a number of other citizen science projects involving the analysis of data packages by PCs and games units around the world (Anderson, 2004). These ‘distributed computing’ projects as they have become generally known, provide a venue where

3SETI@home project homepage: http://setiathome.ssl.berkeley.edu/ 4 This website provides daily statistics on the output and the number of participants in SETI@home and other distributed computing projects: http://www.teamocuk.co.uk/index.php?s=8f315d852c601368eb111539388a9393 5 interested individuals can become involved in scientific research via the Internet (Carroll et al., 2005).

Shortly after the introduction of distributed computing projects such as SETI@home, a new project appeared that asked participants to take a more active part in the analysis of scientific data. This was a small experimental project created by three NASA scientists in

2000 that used volunteers (nicknamed ‘clickworkers’) for scientific tasks that required

“human perception and common sense” (Kanefsky et al., 2001). The tasks did not require a scientific background and involved the identification and classification of craters on

Mars from images taken by NASA’s Viking Orbiter.

Stardust@home5 was the next major project to emerge that enlisted the help of volunteers to take an active role in the analysis of scientific data via the Internet

(Westphal et al., 2006). In 1999 NASA launched the Stardust Mission, in which particles from Comet Wild 2 were collected in the spacecraft’s special aerogel collectors which were parachuted back down to Earth after the completion of the mission. In addition to comet particles, the aerogel collectors may have also captured inter-stellar star dust. In

2006 a project was set up in which participants searched images of the aerogel for signs of inter-stellar star dust. Several thousands have taken part, and the Stardust@home project continues today. Projects such as Clickworkers and Stardust@home were the inspiration for many of today’s largest online citizen science projects such as Galaxy Zoo6 and Planet Hunters7 (Keel 2010, Schawinski 2011). These projects have been referred to as ‘distributed thinking’ projects, and participants help to classify, annotate or transcribe scientific data (Holliman and Curtis 2014).

5 Stardust@home project webpage: http://stardustathome.ssl.berkeley.edu/ 6 GalaxyZoo project webpage: https://www.zooniverse.org/project/hubble 7 Planet Hunters project webpage: https://www.zooniverse.org/project/planethunters 6

Online citizen science has also been referred to as ‘citizen cyberscience’ (Grey 2009, 2011) or ‘virtual citizen science’ (Wiggins and Crowston 2011, Reed et al., 2013). Unlike more

‘traditional’, ecology-based citizen science projects, in which participants help to collect data, these projects are conducted entirely online and the participant analyses data that is provided by the project scientists. Participation in citizen science can thus take place within the comfort of one’s home, on the way to work, or wherever there is access to the

Internet (McDermott 2011). Online citizen science projects have enabled many thousands of interested individuals to become involved in authentic scientific research from anywhere in the globe with internet connectivity (Bohannon 2005, Carroll et al.,

2005, Alexander 2008, Bohannon 2009, Hand 2010).

There are now well over one hundred online citizen science projects to choose from, both in distributed computing and in distributed thinking8 (Grey 2009, Haklay 2011a, 2011b).

More recently (since 2008) several projects have emerged in which scientific research problems have been re-packaged into online multi-player games (Cooper 2011, Kawrykow et al., 2012, Rowles 2013, Curtis, 2014a). These citizen science games have attracted thousands of participants, and have, like other online citizen science projects, experienced some research success. Indeed, a growing number of online citizen science projects have produced significant results that have been published in the academic literature (Lintott et al., 2009; Khatib et al., 2011, Kawrykow et al,. 2012, Schwamb et al.,

2013). These more successful projects have also attracted a fair degree of attention from journalists and science communicators, which can in turn help to increase the number of individuals taking part (Adams 2012, Borrel 2013, Hodson 2013).

8 SciStarter.org is an online repository for citizen science projects. This is a link to a list of current online citizen science projects: http://scistarter.com/activity/10-Exclusively%20online 7

1.4 Gaps in our understanding of online citizen science projects

Despite the growing number of projects, the many thousands of participants, and the potential for significant research results, some aspects of online citizen science projects are not fully understood. For example, who takes part in these projects? Do they appeal to certain individuals or groups? Some projects have hundreds of thousands of registered participants. Do they all participate in the same way, or to the same extent? Are there other aspects of participation that are important such as online social interaction? What motivates individuals to get involved, and can these motivations change over time? The motivations of the scientists who set up online citizen science projects, and the ways in which they can become involved with communities of online volunteers are also poorly understood.

An in-depth investigation of online citizen science projects would help to provide an insight into these issues and contribute to a greater understanding of this growing online phenomenon. Detailed information about patterns of participation and motivation to participate, would also be useful for scientists considering setting up such a project, and could be helpful in creating projects that attract and retain participants.

Since the introduction and development of online citizen science projects, it has become evident that there are different types of projects, with specific types of tasks and user interfaces. Consequently, a small number of researchers have attempted to classify online citizen science projects. Haklay (2011b) proposes a division of online citizen science projects into ‘volunteered computing’ projects and ‘volunteered thinking’ projects. The former are considered to involve more ‘passive’ participation (the participant’s computer or games unit is doing all of the analysis), while the latter involve more of a direct cognitive input from the participant (such as classifying the shape of a galaxy).

Haklay adds a third category: ‘participatory sensing’, where mobile phones are used to sense the environment (e.g. noise levels, air pollution), or to record and report ecological observations. This data can then be collated and used for research purposes. For example, BirdLog9 is a mobile phone app where users can log sightings of specific bird species. This data is used by scientists studying migratory patterns and geographical distribution of birds. Participatory sensing mainly involves data collection rather than data analysis, although there is potential for these apps to be used more widely by citizen scientists in their own research endeavours (Paulos et al., 2008). For the time being however, participatory sensing appears to have more in common with ecological, contributory projects, than with projects like Folding@home or Galaxy Zoo.

A more appropriate third category for online citizen science projects may be citizen science games (Holliman and Curtis, 2014). While the participant contributes through a stylised games interface, they are involved in the direct analysis of data or in creative problem solving (Cooper et al., 2010, Kawrykow et al., 2012, Lee et al., 2014). Unlike some of the tasks required in ‘volunteer’ or ‘distributed’ thinking projects, the games themselves can be quite difficult to learn, and have extended levels of tutorials that teach the player about the game tools and objectives (Cooper, 2011, Andersen et al., 2012, Lee et al., 2014). In some citizen science games, the task can be quite abstract, and appear removed from the underlying science (Kawrykow et al., 2012). Thus, they offer a distinct approach and ‘package’ from many distributed thinking projects.

I have developed a typology based on the work of Haklay (2011b) for use in this thesis that incorporates citizen science games as a third category (Table 1.1).

9 BirdLog was developed by the Cornell Laboratory of Ornithology http://ebird.org/content/ebird/news/birdlog/. 9

Table 1.1 Proposed typology of online citizen science projects

Distributed computing Participants donate their computing power for the analysis of large volumes of data e.g. SETI@home, Einstein@home, Folding@home

Distributed thinking Participants take part in classification tasks, annotation of objects, or the transcription of data (such as scientific log books or field notes)

e.g. Stardust@home, Galaxy Zoo, Old Weather, Planet Hunters, Eyewire

Citizen science games Players help to solve a scientific research problem through a games interface.

e.g. Foldit, EteRNA, Phylo

All three classifications are connected in that they can be accessed or played wherever there is an internet connection, and by the fact that the participant does not have to provide any data themselves. Thus, all three categories are mediated through technology

(Holliman and Curtis, 2014).

This classification also relates to the general level of difficulty of the project task.

Distributed computing projects are easy to install and run, although they often offer participants the opportunity to learn more about the underlying science or interact with other participants online. Distributed thinking projects require a greater cognitive input from the participant and some of the tasks may require a small amount of training or practice. However, tasks need to be relatively straightforward in order to attract participants (Parsons et al., 2011, Ponciano et al., 2014). Citizen science games appear to have the greatest level of complexity and some require considerable training before the participant is able to make a useful contribution (Andersen et al., 2012).

It has become clear over the course of this research, that the area of online citizen science is developing rapidly, and the above typology may well need to be re-visited in the future. This typology does help however, to make sense of the current state of online citizen science (as of 2014), and has been used as a foundation for this research, particularly in the selection of projects to study. Thus, in order to explore a range of online citizen science projects, a project from each ‘type’ has been investigated in detail.

The aim has been to explore certain aspects of participation, as well as the motivations that initiate and sustain participation. I have considered both the citizen scientist volunteers, and the scientists and developers who set up and manage online citizen science projects.

Exploring multiple projects allows comparisons to be made and also permits an inquiry into how (or if) aspects of participation and motivation are related to types of project tasks, or to the type of project. This thesis will therefore focus on the following research questions:

1. Who participates in online citizen science projects?

2. What motivations initiate and sustain participation in online citizen science

projects?

3. How do motivations vary between different types of online citizen science projects

and their associated tasks?

4. How and why do project participants interact online?

5. How can contribution to online citizen science projects be characterised?

6. How do participants perceive their role in the project?

The following projects were selected as the focus of this research Folding@home (a distributed computing project), Planet Hunters (a distributed thinking project) and Foldit

(a citizen science game). These projects will be described in detail in Chapter Four.

1.5 Organisation of the thesis

In addition to this Introduction, the thesis is presented in eight further chapters.

Chapter Two presents a review of the literature and places this research within the context of previous work. This also includes a consideration of other types of online peer production such as open source software and the production of open content. Gaps in our understanding of how and why individuals participate in online citizen science are identified and six research questions are proposed. Models relating to motivation and patterns of participation are considered. Of particular relevance are Preece and

Schneiderman’s (2009) ‘reader-to-leader’ framework and Haythornthwaite’s (2009) model of ‘lightweight and heavyweight’ modes of peer production.

Chapter Three discusses the methodology and methods used to address the research questions. A mixed methods approach that utilises online surveys, semi-structured interviews and participant observation is outlined, as is the predominantly qualitative analytical approach.

Chapter Four provides a brief background for each of the three online citizen science projects that have been selected as a case study. The research objective, project task, as well as other project parameters (e.g. opportunities for interaction, and the numbers of active participants) is described. This information has been collected through my observations and participation in the three projects. Each project section will conclude

12 with an overview of my experience as a participant, and create a context within which to consider the results of the surveys and interviews.

The results for each selected project are presented separately and the findings from online surveys, and semi-structured interviews are outlined. The projects are presented in the order in which their investigation commenced. Chapter Five presents the Foldit results, Chapter Six presents the Folding@home results, and Chapter Seven presents the results from Planet Hunters.

Chapter Eight presents a comparative analysis of these findings. The projects are compared within the context of the research questions, and within the overarching themes of who participates, why they participate, and how they participate. Possible explanations for the findings are discussed, and the relevance and utility of the models outlined in Chapter Two are considered.

Chapter Nine concludes this thesis by examining the contribution of this research in detail and in relation to the research questions. Potential limitations of this study are considered, and future avenues of related research are proposed. A general timeline of all research activities is presented in Figure 1.1.

Figure 1.1 Timeline of research activities

Chapter 2: Review of the Literature

2.1 Introduction

This literature review will focus on aspects of participation in online citizen science projects. It will consider previous work that has examined the demographic characteristics of those who take part in online citizen science projects, and a small body of work that examines how citizen scientists contribute to projects and how they interact online. I will also review the current literature focussing on motivation to participate in online citizen science, and also in other types of voluntary activity that may be of relevance to this research. My research questions will be presented within the relevant sections of this review.

As online citizen science is a newly-emerging area, there are multiple sources of information in relatively disparate fields. My search for relevant literature has utilised mainly electronic means of retrieval allowing for multiple searches across disciplinary boundaries. I have used the search engines of the Open University online library catalogue and also the University of Cambridge Library. I have made use of Google

Scholar and set up electronic alerts for a number of key words such as ‘citizen science’,

‘distributed computing’ ‘crowdsourcing’ and the names of the three projects I have investigated. These alerts have enabled me to keep track of new publications throughout the research period. Key journals in areas of interest (e.g. science communication journals, journals in disciplines where citizen science is carried out, and some computing journals) as well as a number of relevant blogs by researchers in related areas were identified and regularly monitored. In addition to electronic searches, I have undertaken

‘hand searches’ of the literature (browsing titles upon the shelves) at the Open University, and many useful references have ultimately been found within the bibliographies and

15 references cited by other researchers working in related areas. The literature has been constantly monitored throughout the period of study and more in-depth reviews were undertaken approximately every 6-8 months (see Chapter One, Figure 1.1). The content of online searches and the alerts has changed throughout the course of my study, as the research questions and areas of focus have evolved and developed.

2.2 Main features of online citizen science

Online citizen science projects have a website that serves as the public interface of the project10. They provide background information such as the specific research aims, and educational material so that participants may learn more about the related science.

Many online citizen science projects contain forums where participants can interact with each other and (in some cases) with the project scientists or developers. Online forums enable participants to discuss the project, share problems they may be having with the tasks, offer help to others, or ask questions of the scientists. Some project forums also have areas where more general topics (unrelated to the science or the project) can be discussed such as current affairs, or hobbies and interests11. Having an ‘area’ that allows interaction between citizen scientists, and between citizen scientists and the project scientists, can provide opportunities for co-operation and collaboration between project participants relating to project tasks (Paulus 2005).

Online citizen science projects consist of tasks that have ‘granularity’, that is, the work consists of much smaller units that can be easily distributed among the participants (Nov et al. 2011). For example, a unit could be an individual work ‘package’ in a distributed computing project, or the classification of a single object in a distributed thinking project.

10Examples of project homepages can be seen for Foldit http://fold.it/portal/; Cosmoquest http://cosmoquest.org/; and Old Weather https://www.zooniverse.org/project/oldweather. 11 This area of the SETI forum is devoted to discussion about politics: http://setiathome.berkeley.edu/forum_forum.php?id=23. 16

Individual tasks, may not take very long to complete which means that participants can make a contribution whenever they have a small amount of free time or are in between other activities. This ability to make small contributions has been referred to as

‘microvolunteerism’ and has been key in the success of a number of online citizen science projects as it allows individuals the flexibility to vary the amount they contribute during any one visit, and to tailor their contribution according to other commitments (Paulos et al. 2011).

In many cases, the tasks given to citizen scientists are straightforward and do not require any pre-requisite academic qualifications in science. By keeping tasks relatively simple, projects may be more accessible to a wider variety of individuals with differing levels of interest or ability (Parsons et al. 2011), and also increase the likelihood that the work carried out by citizen scientists is accurate and reliable (Cohn 2008). However, the complexity of the task can vary between different online citizen science projects and training on how to perform the task may be required through an online tutorial.

Feedback can also be given to participants when they are taking part in the project and in some cases ‘dummy’ tasks are given to participants in order to remind them of the original task parameters12. In citizen science games, the required task can be quite complex, and project scientists and developers in some games (such as Foldit and

EteRNA) have put together a series of tutorial puzzles that introduce the tools and concepts behind the game (Andersen, O'Rourke et al. 2012). How the level of task complexity relates to aspects of participation such as participant numbers, level of contribution, motivation to participate and how participants interact online has not previously been investigated in much detail and will be explored in this research.

12 In Spacewarps ‘dummy’ tasks are given at intervals to participants (see: http://spacewarps.org/) 17

In more ‘traditional’ citizen science projects where the participant is involved in data collection, the accuracy of data provided by citizen scientists has been much discussed in the literature (Cohn 2008; Dickinson, Zuckerberg et al. 2010; Newman, Crall et al. 2010).

How reliable citizen scientists are in the tasks assigned to them in online citizen science projects, has not been discussed as widely. However, some online citizen science projects have tried to address this problem by having more than one citizen scientist complete each task. For example, in projects where citizen scientists are involved in classification tasks (such as Galaxy Zoo and many other projects in the Zooniverse), more than one individual will classify the same object (Schawinski 2011; Schwamb, Lintott et al. 2014).

The ‘majority vote’ is in operation here, and the classification with the greatest level of agreement is accepted. This is more readily achieved in projects where there are hundreds or thousands of participants.

2.3 Who takes part in online citizen science?

Citizen science has on occasion been written about within the context of open science, and with a move towards a greater ‘democratisation’ of science – suggesting that scientific research is accessible to anyone who wishes to take part, even if they lack the formal educational qualifications (Könneker and Lugger, 2013, Nielsen, 2012, Stodden,

2010). However, there is little data available about who is taking part in citizen science projects. Information regarding the demographic characteristics of participants in ecology-based projects must be gleaned from a small number of published studies that provide limited information about sub-samples of the overall population of project participants (Rotman et al., 2012, Trumbull et al., 2000). One of these studies (Rotman et al. 2012) noted a greater percentage of male volunteers (57%) in a sample of their participants. A study by Trumbull et al. (2000) observed that volunteers generally had an

18 interest and positive views toward science, and that 70% of a sample of their participants were educated to at least an undergraduate degree level.

There is slightly more demographic data available for online citizen science projects and

Table 2.1 summarises the demographic information currently available from seven projects (SETI@home, 2006, World Community Grid, 2013, Estrada et al., 2013, Holohan and Garg, 2005, Krebs, 2010, Raddick et al., 2010, Reed et al., 2013). Much of the information about the demographic characteristics of those taking part in online citizen science projects, are from distributed computing projects.

Table 2.1: Demographic data of citizen scientists obtained from 7 published studies

Author / year Project and sample size Demographic details of sample Holohan and Garg, 2005 Various distributed computing 98.4% were male, and most aged projects including SETI@home between 26 and 49. 70% based in USA and GIMPS (Great Internet and Canada, and 24% based in Europe. Mersenne Prime Search) n=323 SETI@home team, 2006 SETI@home distributed 92.74% are male, and 61% were aged 20- computing project, n=142 000 39. Krebs, 2010 malariaControl.net distributed 56% of participants were based in Europe computing project, n ranges and 33% in North America. Most were from 693 -1097 aged between 20 and 50. 87.8% were male (n=693). Most of the survey participants were IT professionals. Estrada et al., 2013 Docking@home distributed 80% were male, and most males were computing project n=739 aged between 31and 35. Female respondents were aged mainly between 46 and 55. Small representation of ‘ethnic minorities’. World Community Grid World Community Grid 90% of sample was male, and most have member study, 2013 collection of distributed a “technical knowledge base”. Most aged computing projects, n=15 627 between 25 and 44. 36% work in information technology. Reed et al., 2013 Zooniverse projects, n=199 67.3% were male, with a mean age of 40.7. Most based in USA or UK. Many had a college degree (119 participants provided this info on education). Raddick et al. 2013 Galaxy Zoo, n= 10 708 82% are male, and the mean age is 43.2 with no clear age trends. Most respondents are from North America and Europe. Over half have at least a bachelor’s degree.

These studies suggest that online citizen science projects may attract more male than female participants. Previous attitudinal research carried out in the UK suggests that men may be more engaged with science than women, taking a greater interest in science- related issues and participating in more science-based activities (RCUK, 2008). The underrepresentation of women in computing-related academic disciplines and professions has also been well documented (European Commission, 2012, Camp, 2012,

Klawe et al., 2009), and this may account for the small proportion of female participants in distributed computing projects.

Two of these studies have also shown that participants may be well educated with significant proportions having a tertiary-level education (Reed et al., 2013, Raddick et al.,

2013). The survey of World Community Grid participants illustrated that many have a

‘technical knowledge base’ and work in IT-related professions (World Community Grid,

2013). Overall, there is little information regarding those who participate in distributed thinking projects, and no information regarding those who play citizen science games.

However, the degree to which the samples in these studies are representative of the wider group of project participants is not always clear. In some of these studies, the total number of project participants is not reported. This is the case for the Holohan and Garg study (2005), The World Community Grid survey (2013) and for the SETI@home survey

(2006). There is also a discrepancy in how overall project numbers are reported. For example, in the Estrada et al. study, the total number of registered participants is provided (estimated to be 27 000 in Docking@home), while Krebs (2010) gives the number of active participants (estimated to be 10 000 for MalariaControl.net). Raddick et al. (2013) report that the total number of registered users of Galaxy Zoo was in the region

20 of 175 000, while the study of Reed et al. (2013) does not specify which Zooniverse project study participants were recruited from.

The concepts of ‘registered’ participants and ‘active’ participants will be considered in more detail in Section 2.5 and in Chapter Four, as it is likely that that this distinction is of relevance to online citizen science projects. For example, what proportion of registered participants actively contribute to a project, or regularly undertake project tasks?

Previous work exploring other online communities has demonstrated that not all those who register an initial interest in a project will become active contributors (Kittur et al.,

2007, Shirky, 2009, Preece and Schneiderman, 2009, Clow, 2013).

While the existing demographic data is of interest, a greater understanding of who participates in these projects may help to shed light on whether online citizen science can offer increased opportunities for wider participation in scientific research, or whether their appeal is more restricted. More information on participants may be of interest to those thinking about establishing an online citizen science project, particularly if they have educational or public engagement goals as well as research goals. The lack of data in this area, as well as the potential relationship with other aspects of participation (such as the level and type of activity in a project, or the motivation to participate), have formed the basis of the first research question.

Research question 1: Who participates in online citizen science projects?

This research will aim to add to the existing body of data by extending the range of online citizen science projects that are explored in this way, and by asking a wider range of questions of participants. In addition to data relating to age, sex, educational background, and occupation, I will also explore general level of interest in science and

21 participation in other science-based activities, something which has not been considered in previous work relating to citizen science.

2.4 Motivation to participate in online citizen science projects

For anyone setting up an online citizen science project, it is important to know how to attract and retain participants if the tasks are going to be completed and the research goals are to be achieved (Reed et al., 2013, Rotman et al., 2012). For example, what makes a project attractive and interesting for a potential participant? How can interest and participation be maintained? Understanding participant motivation is key to the long-term success and sustainability of a project.

Hundreds of thousands of individuals have registered to participate in citizen science projects, and they do so without any monetary re-numeration. It is essentially a type of volunteerism (Sproull, 2011). Research has been carried out on the motivations of those who volunteer generally and carry out charitable work (Houle et al., 2005, Hustinx et al.,

2010), while a small number of studies have examined the motivations of those who take part in commons-based peer production such as open-source software production and editing for Wikipedia (Hertel et al., 2003, Kuznetsov, 2006). A consideration of this work may help to shed light on why people take part in online citizen science projects, and may provide some frameworks with which to consider this area further.

2.4.1 Motivation and volunteerism

Volunteering can be defined as discretionary behaviour such as assisting, comforting, sharing and co-operating intended to help people other than oneself (Wilson, 2000). It has been further described as prosocial, non-obligated by family or friendship, and situated within an organisational context (Sproull, 2011). A number of frameworks have been developed to explore motivation for voluntary behaviour. Two in particular (see 22

Section 2.4.5) have been considered in relation to citizen science (Rotman et al.,2012,

Raddick et al., 2013).

Work by Clary et al. (1998) suggests that people are motivated to volunteer because it fulfils certain functions that reflect important features of self and identity. For example, people choose an activity because it allows them to express the values that are important to them. In this model, volunteering serves six potential functions.

1. Values: volunteering allows individuals to express values related to their altruistic

and humanitarian concern for others.

2. Understanding: volunteering provides an opportunity for new learning

experiences and to exercise knowledge and skills that may otherwise go

unpractised.

3. Social: volunteering provides opportunities to be with one’s friends and peers, or

engage in an activity that is viewed favourably by others.

4. Career: volunteering may provide career-related benefits (i.e. the development of

new skills, leadership opportunities).

5. Protective: volunteering may protect the ego from negative features of the ‘self’,

and may serve to reduce guilt over being more fortunate than others, or to

address one’s ‘personal problems’.

6. Enhancement: this function involves a motivational process that centres on the

ego’s growth and development (e.g. some people volunteer for reasons of

‘personal development’).

Volunteers who serve in roles that match their own motivations will derive more satisfaction and enjoyment from their service and are more likely to continue (Clary et al.,

1998). 23

After studying ‘community involvement’ Batson et al. (2002) concluded that motives arise in a given situation and are a function of the values of the individual and the nature of the situation. Furthermore, motives can change over time – often quite quickly (Batson et al.,

2002). They define motives as “goal-directed forces induced by threats or opportunities related to one’s own values” (p. 430). A goal can be either ultimate or instrumental. An ultimate goal is the valued state the individual is seeking to reach, while the instrumental goals are sought as they act as stepping stones to one’s ultimate goals. These are related to four different underlying drivers:

1. Egoism: where one acts to increase one’s own welfare.

2. Altruism: the ultimate goal in this instance is the increased welfare of others, apart

from oneself. The source of this motivation is empathic emotion and extends to a

group where one may be a member. Not all groups invoke equal empathy.

3. Collectivism: when an individual is motivated by the increased welfare of a group

or collective. This is directly focussed on the ‘common good’, but may be limited

to one’s own ‘in-group’.

4. Principlism: where one is motivated by the ultimate goal of upholding some moral

principle such as justice.

Different motivations interact and do not always work in harmony, and motives to promote the welfare of self or group can undercut or compete with one another.

2.4.2 Motivation and other types of activity

There is a substantial body of work on motivation. While some has focussed on general volunteering, some work has been carried out in the area of formal education by Ryan and Deci (2000, 2009). This work has been applied to disciplines other than education, and has informed a previous study of motivation to take part in a distributed computing 24 project (Krebs 2010). Their work is based on self-determination theory which defines the natural tendency to learn and develop as something that is influenced by the inner world of drives, needs and experiences (Ryan and Deci, 2009).

In order to be motivated, an individual needs to be moved to do something. These motivations can be either intrinsic or extrinsic (Ryan and Deci, 2000). Intrinsic motivation involves carrying out an action because it is inherently interesting or enjoyable. When intrinsically motivated, an individual is moved to act for the fun or challenge of an activity, rather than because of external ‘prods’, pressures or rewards. Extrinsic motivation is engaged when doing something leads to a separable outcome such as a reward, or a desirable reaction from a significant other to whom they feel (or would like to feel) a connection (e.g. family, peer group, society). Extrinsically motivated behaviours are not always inherently interesting and must be prompted.

The motivation to join social movements (e.g. the peace movement in the 1980’s) has been explored by Klandermans (2003), and has formed the basis of a theoretical framework that has been considered by one previous study exploring motivation to participate in online citizen science projects (Nov et al., 2011b). In this model there are four main types of motivations.

1. Collective motives, where someone is motivated to join a movement because of

the importance they attribute to the project’s goals.

2. Norm-oriented motives, where participants are motivated by the expectations or

reactions of significant others such as family, friends or colleagues.

3. Reward motives, or the benefits that one can gain as a result of participation such

as making friends or gaining reputation.

4. Collective identification, when individuals identify with the social group and its

practices.

These frameworks have been applied in some of the previous work exploring the motivation to participate in online citizen science projects, and will be considered in greater detail in Section 2.3.5. The use and relevance of motivational frameworks to this study will be explored further in the analysis and discussion of the data (Chapters Eight and Nine).

2.4.3 Motivation and commons-based peer production

A small number of researchers have likened some online citizen science projects

(particularly distributed computing projects) to other types of online collaborations such as the production of open-source software, or the production of open content such as

Wikipedia (Benkler and Nissenbaum 2006; Shirky 2009). These types of projects have become known as ‘commons-based peer production’, a term first coined by Benkler

(Benkler 2006). Commons-based peer production is made possible by the Internet and involves the collaboration of large numbers of people to provide information, knowledge or cultural goods without relying on economic factors or an over-riding management structure or hierarchy. Commons-based peer production is also highly granular and smaller tasks are allotted to distributed participants. Tight-knit online communities working towards a common purpose may emerge (Kreiss, Finn et al. 2011).

A small body of work exists in relation to the motivations of those who produce open source software, and those who write articles for shared content websites such as

Wikipedia. Like online citizen science participants, open-source software writers and

Wikipedia contributors are not generally paid to produce content and their motivations for doing so have been attributed by researchers in this area to elements of both altruism 26 and egoism (Hars and Shaosong, 2002, Chang and Yang, 2009). Some of this work on common-based peer production has also utilised some of the motivational frameworks considered by researchers looking at online citizen projects.

Studies examining the motivation of Wikipedia writers / editors have shown that they tend to be motivated by more altruistic reasons, which are often based upon the belief that information and knowledge should be freely available to anyone (Forte and

Bruckman, 2005, Kuznetsov, 2006, Nov, 2007). Unlike open-source software, Wikipedia has no established public recognition system that reflects individual contributions

(Schroer and Hertel, 2009). However, the history of every article is available and authors often claim ownership of their articles and keep lists of their contributions. Within communities of Wikipedia editors, this may serve as a system of recognition and is therefore an extrinsic motivation (Ciffolilli, 2003). Schroer and Hertel (2009) looked at

Wikipedia as a social movement and used the framework of Klandermans (2003) to characterise motivations to participate. While Wikipedia does not focus on political protest, there is a common underlying philosophy associated with the goal of free knowledge for everyone (Nov, 2007).

Open-source software contributors on the other hand, are often motivated by more egotistical concerns such as establishing a reputation as a competent coder and the securing of employment opportunities (Hars and Shaosong, 2002, Lakhani and Wolf,

2005). The development of open-source software involves a review system that is similar in some ways to the academic peer review system, and software is released only if it is deemed good enough by the reviewers (Oreg and Nov, 2008). However, altruism and an ideology centred on the free provision and access to software solutions are also strong motivations for many (Hertel et al., 2003, Lakhani and Wolf, 2005, Oreg and Nov, 2008). 27

Like Wikipedia, open-source software has parallels with other social movements, and a strong ideological core has also been observed within these communities (Hertel et al.,

2003).

Some recent work on OpenStreetMap, a website where participants provide local geographic information that can be shared and edited, suggests that motivation to participate varies between those classified as ‘casual’ mappers, and those considered

‘serious’ mappers (Budhathoki and Haythornthwaite, 2013). Casual mappers were motivated by a belief in the general principle that mapping data should be freely available, while more serious mappers (those defined as ‘core’ and repeat contributors) were more orientated to community, learning, local knowledge and career motivations.

The observation that motivation can vary depending on type of contribution is of interest, and may have implications regarding factors motivating participation in online citizen science projects (Crowston and Fagnot, 2008).

2.4.4 Motivation and citizen science

Despite the fact that contributory ecology-based citizen science projects have been around for many decades, and that hundreds of projects have been undertaken, there are only a small number of studies that have explored why people are motivated to participate. Two of these studies are based on conservation volunteers, and both found a desire to help, and interest in conservation issues were important motivators for participants (Bradford and Israel, 2004, King and Lynch, 1998).

One of the more comprehensive studies to look at motivation and citizen science not only explored motivations for joining a number of conservation-based projects, but also explored motivations for remaining with the project (Rotman et al., 2012). Using the motivational framework of Batson et al. (see Section 2.3.1) as a guide, Rotman et al. 28 found that motivation was dynamic and temporal in nature. For example, an important primary motivator was a personal interest in the project combined with an interest in gaining something from the project (such as skills). These initial motivations are based on egoism, in that there is something to be personally gained from participation.

As participants became more involved with the project over time, secondary motivations became more important. For example, factors relating to community involvement, and the opportunity to develop a better understanding of conservation issues were key in sustaining their involvement with the project. These motivations are based more on collectivism and altruism (Batson et al., 2002). Another important secondary motivator was recognition and attribution, and participants wanted to be given the appropriate credit for the work they had done. The temporal nature of motivation is of relevance considering that many citizen science projects (both traditional and online) can continue for many months or even years (Lintott et al., 2008, Cooper, 2011). Those setting up citizen science projects must therefore take into consideration these changing motivations during the design process and when formulating the project tasks (Prestopnik and Crowston, 2012).

Another observation of note in this work was that motivations could be affected by the attitude of the project scientists towards the citizen scientists. For example, some participants saw their involvement in the project as an opportunity to improve and extend their knowledge of specific habitats and species. This motivation appeared to be stronger where volunteers had greater contact with scientists in the field, and when educational opportunities were facilitated and encouraged. This is the only study located, where the motivations of the project scientists has been considered (Rotman et al.,

2012). This group also concluded that Batson’s model of motivation did not translate well

29 for citizen science projects as the projects are complex, involve many different tasks.

However, the model of Batson et al. (2002) is of relevance here as it highlights the temporal nature of motivations.

2.4.5 Motivation and online citizen science

While the motivations behind general volunteering, participation in commons-based peer production and ecology-based citizen science projects may provide some insight as to why people take part in online citizen science projects, it is important to note that these projects occur within a context which may be quite different. Online citizen science could be viewed as more opportunistic (e.g. there is flexibility with regard to time and place of participation unlike working for a charity, or collecting ecological data), and the projects may be more accessible in that they do not necessarily require specialist skills or knowledge to make a contribution (unlike writing a piece of open-source code, or an entry for Wikipedia).

Participants in online citizen science projects are usually working within the research protocols and procedures imposed upon them by the project scientists, and do not organise themselves as in open software or open-content communities (Reed et al.,

2013). All of these factors may have an influence on motivation; therefore, it is important to consider studies that have specifically looked at online citizen science. Studies that have reported findings relating to motivation of participants in online citizen science studies are listed in Table 2.2 along with the methodology employed to collect data. Four of them have focused on distributed computing projects.

Table 2.2: Studies exploring motivation in online citizen projects

Author / year Project Data collection method

Holohan and Garg, 2005 Distributed computing projects Online qualitative Survey (n=37) including SETI@home and Online quantitative survey (n=323) GIMPS (Great Internet Mersenne Prime Search) Nov et al., 2010 SETI@home Online quantitative survey (n=274) Krebs, 2010 malariaControl.net and other Online quantitative survey using Likert distributed computing (BOINC) scales (MalariaControl.net, n=1097; projects BOINC projects, n=408) World Community Grid World Community grid collection Online quantitative survey (n=15 627) no member study, 2013 of distributed computing details of survey format, followed by 6 projects, n=15 627 focus group sessions (n=unknown) Nov et al., 2011a Stardust@home, n=139 Online quantitative survey using Likert scales Raddick et al., 2010 Galaxy Zoo, n=22 Qualitative Interviews (via Skype or instant messenger), plus the confirmation of motivations in 826 forum posts Raddick et al. 2013 Galaxy Zoo, n= 10 708 Online quantitative survey using Likert scales

How representative these sample sizes are is unclear, and the same issue that relates to the number of ‘registered’ vs. the number of ‘active’ participants as outlined in Section

2.3 in relation to Table 2.1 (demographic characteristics of citizen science participants) is of relevance here. Four of these studies are also included in Table 2.1 (Holohan and Garg,

2005, Krebs, 2010, World Community Grid, 2013, Raddick et al., 2013). The earlier study by Raddick et al. (2010) forms the basis for the later study, and is also drawn from a sample of 175 000 registered Galaxy Zoo participants. The studies on SETI@home (Nov et al., 2010) and Stardust@home (Nov et al., 2011a) do not report the total number of either registered or active participants, although the Stardust@home survey was sent to participants who had been active during the previous 30 days, which is perhaps an attempt to reach this group of active participants.

Holohan and Garg explored motivations in SETI@home and GIMPS (Great Internet

Mersenne Prime Search)13 participants as part of a wider study that looked at online collaboration (Holohan and Garg, 2005). Results relating to motivation suggested that survey participants were mainly motivated by making a scientific contribution, followed by the competition with other participants (those who take part in distributed computing projects are awarded points for each work unit their computer processes). Other important motivators were the social aspects of participation (interaction with other participants and the sense of community) and the opportunity to gain greater technical knowledge about computers.

Nov et al. (2010) conducted a survey of 274 randomly selected SETI@home participants.

They linked survey findings on motivation to actual levels of participant contribution as determined by their activity logs (Nov et al., 2010). They considered intrinsic and extrinsic motivations, and also whether these motivations were ‘self-oriented’ or ‘project oriented’. They found that ‘self-oriented’ factors relating to personal enjoyment and enhancement of reputation were important motivators, but that this was not statistically related to contribution levels. However, being affiliated to a team was positively related to contribution levels, and this result may suggest that being part of a social structure is important to these participants (Nov et al., 2010). This study is the only one identified so far that tries to link motivation with contribution levels. However, this is only measured in terms of work units completed (and the resulting number of points). This thesis will explore the relationship between motivation and contribution in more detail, and will also consider other types of contribution in addition to the completion of project tasks, such as participation in online discussions and forums (see Section 2.6).

13 The Great Internet Mersenne Prime Search uses distributed computing to find Mersenne prime numbers. A Mersenne prime is a prime of the form 2P-1. Forty-six have been discovered so far (http://www.mersenne.org/). 32

Krebs (2010) looked at the motivations of participants in MalariaControl.net, a project where participants’ computers run epidemiological models of malaria infection. She also looked at the motivations of a smaller group of participants who participate in a number of other BOINC14 distributed computing projects (Krebs, 2010). Using a previously developed list of 10 potential motivations (that were either intrinsic or extrinsic), Krebs found that wanting to contribute to a community (also referred to as ‘solidarity’), and getting involved in a particular cause were the most important motivators for

MalariaControl.net participants. These two motivations were also the most important for the group of general BOINC participants. Less important motivators for

MalariaControl.net participants were more extrinsic motivators relating to enhancing professional experience, networking, learning and knowledge sharing.

These more extrinsic motivations have been found to be of greater importance in studies looking at those who write open-source software (Hertel et al., 2003, Lakhani and Wolf,

2005, Oreg and Nov, 2008). This study also suggests that recognition of a volunteer’s contribution is important and many respondents stated that it was important to provide tangible rewards (e.g. points). Respondents further highlighted the importance of feedback and communication from the organisers, and knowing what went on behind the science. Survey feedback also emphasised the importance placed on the social component of the project as some participants like to interact with others. However, the degree of interaction between participants, or between the participants and the

‘organisers’ was not explored in any detail. Nor was motivation was linked to level of contribution.

14 BOINC stands for Berkeley Open Infrastructure for Networked Computing. BOINC software (or ‘middleware’) is now used in most distributed computing projects. It is based on the software originally developed for SETI@home. 33

Motivation to participate was explored in a recent survey of participants in the World

Community Grid, a collection of distributed computing projects overseen by the IBM

Corporation that focus mainly on humanitarian issues such as cancer epidemiology, the search for disease biomarkers, and computational analysis of potential drugs for HIV/AIDS

(World Community Grid, 2013). This survey of over 15 000 participants found that the main reason respondents participated (69%) was to support scientific research with goals that they believed were important. The next most important reason (cited by 58%) was to make use of their unused computing power. The latter motivation may be a reflection of the fact that a large proportion of respondents have a ‘technical knowledge base’ (over a third of respondents worked in information technology). The fact that all the results generated through the World Community Grid are publicly available was also important to some respondents. Similarly to the MalariaControl.net study, World Community Grid participants also wanted more information about the impact of their contributions and regular updates from the scientists and researchers involved in the projects.

These studies examining distributed computing do not take into account the involvement of communities of computer hardware enthusiasts, sometimes known as ‘overclockers’.

Overclockers build and design their own computers for maximum work output, and often use distributed computing programmes to test or benchmark their machines (see Chapter

Four, Section 4.2.3). These communities are important contributors to distributed computing projects and may account for over 50% of project outputs (Bohannon, 2005).

This community has not been well-researched and their motivations may well be different to other participants. I will consider the contribution and motivations of overclockers in my own research on a distributed computing project (Folding@home).

In addition to studies on distributed computing projects, work on participant motivations has been carried out on two distributed thinking projects: Stardust@home and

GalaxyZoo. The same group involved in exploring motivations in SETI@home participants, has also explored the motivations of Stardust@home participants and related them to levels of contribution (Nov et al., 2011a). On this occasion, a framework based on

Klanderman’s framework of voluntary participation in social movements (see Section

2.3.1), was used to categorise motivations.

Survey results of 139 participants showed that ‘collective’ motives (i.e. the importance attributed to the project goals) and ‘intrinsic’ motives (i.e. the enjoyment of taking part) were the most commonly cited reasons for taking part. Intrinsic motivation was also found to be associated with ‘participation effort’ and the more the participant enjoyed the experience of taking part in Stardust@home, the more they contributed. However, most respondents (over 100) participated for less than two hours a week. The appropriateness of Klandermans model was not discussed in detail nor was the extent to which participation in online citizen science projects could be considered as a parallel to participation in a social movement.

In a further analysis, this group compared the motivations of these participants in

Stardust@home with a larger sample (1843) of SETI@home participants, and explored whether task granularity was related to motivation (Nov et al., 2011b). They defined low granularity tasks as more ‘passive’ involving less participant input, and this included running a distributed computing programme. Higher task granularity was defined as more ‘active’ and would include tasks such as image classification or analysis (such as the

Stardust@home task). Results of this study suggest that task granularity is positively

35 correlated with motivation levels, but that additional research was needed to determine the direction of this relationship.

Examining motivation in relation to actual contribution and to task granularity has not been considered by other research groups, and the findings of Nov et al. may have implications for the designers and managers of online citizen science projects. This research will further explore the relationship between motivation and contribution, and will consider other ways in which a participant may contribute to a project in addition to the main project task.

Raddick et al. (2010, 2013) have carried out two studies exploring the motivations of

GalaxyZoo participants. GalaxyZoo was one of the first distributed thinking projects and involves the classification of galaxies according to a number of easy-to-recognise physical characteristics. In the first study, the group interviewed 22 participants to explore their motivation for participating. Instead of using an existing framework to consider motivations they used a grounded theory approach and devised a list of 12 important motivators such as ‘astronomy’, ‘community’, ‘contribute’ and ‘discovery’ based on the interview feedback (Raddick et al., 2010). One important observation was that each participant had several reasons for taking part in the project. In this small group of participants, an interest in astronomy, and the desire to contribute to the project were among the most important motivators. Feedback to a forum posting asking why individuals were taking part were also examined, and the same two motivations were also found to be the most important among the 826 responses.

The list of 12 motivations derived from participant feedback during the first study was further explored using a larger group of participants. A more detailed follow-up survey was constructed utilising Likert scales, and over 10 000 GalaxyZoo participants responded 36

(Raddick et al., 2013). The most important primary motivation for participation was making a contribution to science (this amounted to nearly 40% of the responses), and an interest in astronomy (just over 10% of responses). The possibility of making an important discovery was also important for about 10% of respondents.

This small collection of studies has shed some light on what motivates online citizen science participants, but there are no detailed studies that examine the motivation of those who take part in citizen science games such as Foldit or EteRNA. Although a small study looking at the potential of games to attract participants to citizen science, found that four Foldit participants were attracted to the game because they were interested in science more than they were interested in games (Iacovides et al., 2013).

The number of citizen science games has gradually increased over the past five years, and one game in particular – Foldit, has had some success in terms of scientific advances directly resulting from the work of project participants (Eiben et al., 2012, Khatib et al.,

2011b). A growing number of research groups and learned societies are exploring the use of science-based video games for informal science learning and public engagement

(Curtis, 2014b). Also, the proportion of the population in the UK and USA who play video games is growing (Anderson and Rainie, 2012; IAB, 2011). Therefore, more work examining the motivations of those who take part in this type of project would be of interest, and may illuminate what approaches, features and designs work well in attracting and retaining participants. Using elements of video games, or ‘gamifying’, has been applied to other types of online citizen science projects (e.g. Eyewire15) with some

15 Eyewire is a project that enlists the help of volunteers to help map the 3D structure of neurons. It uses elements from video games both in its graphical interface, and in its use of competition and leader boards (see http://blog.eyewire.org/about/) 37 success. Through a detailed examination of the citizen science game Foldit, this research aims to address this gap in knowledge.

There has been some consideration of whether participant motivations remain constant over time, and if the factors that draw an individual to a project, are the same that sustain their interest over months or years (Rotman et al., 2012). This area is of interest, and has implications for those considering setting up an online citizen science project and for a project’s sustainability. However, most of the detailed work in this area was carried out in contributory ecology-based projects (Rotman et al., 2012) and further exploration of this phenomenon is needed in online citizen science projects, particularly as previous research has shown that some online citizen science projects have a high attrition rate

(Nov et al., 2011b, Ponciano et al., 2014). My research explores motivation to remain with an online citizen science project as well as the motivation to join.

In addition to the motivations of citizen scientists, the motivations of professional scientists and developers who set up online citizen science projects are not well understood. Only one study has been found that has considered the motivations of professional scientists to take part in citizen science projects, and this was the study of

Rotman et al. (2012). They surveyed and interviewed scientists involved in ecology-based citizen science research and found that many saw these projects as a way to facilitate large-scale data collection. Furthermore, many of the scientists preferred to have volunteers limited to the role of data collectors, while they asserted themselves as the leaders of the research. While data collection was the primary motivation, the secondary motivations were more altruistic, and scientists saw these projects as a vehicle for educating members of the public. The authors of the study state that understanding the motivations of both scientists and citizen scientists, may help to facilitate “broader,

38 sustainable, and more inclusive collaboration between scientists and volunteers” (Rotman et al., 2012, p 225).

To explore motivations in a wider variety of online citizen science projects, and to build further on the work of Rotman et al. regarding motivations that sustain participation, the following research question will be addressed.

Research question 2: What motivations initiate and sustain participation in online citizen science projects?

Not only will the motivations of citizen scientists be explored, but also the motivations of scientists and developers who are involved in setting up and managing these projects.

The views of this group have not been previously explored in online citizen science projects. The utility of the motivational models outlined for considering motivation to participate in online citizen science projects will be considered in Chapter Eight.

Only one previous study (Nov et al., 2011b) has compared the motivation to participate in two online citizen science projects. This research will compare motivation to participate in three different projects. Motivation to join as well as motivation to remain participating in a project will be compared. The work of Nov et al (2011b) on

Stardust@home and SETI@home found that task granularity may be related to motivations to participate (Nov et al., 2011a). This is a notable finding, and a further exploration of this phenomenon in relation to the complexity of the project task and the availability of other related tasks may help to further understand motivations for participation. To explore this finding further the following research question will be addressed.

Research question 3: How do motivations vary between different types of online citizen science projects and their associated tasks?

In order to address this question the nature of the task associated with each project, as well as other opportunities for contribution have been considered in detail.

2.5 Interaction in online citizen science projects

Observations of online citizen science projects have shown that interaction between participants can occur in a variety of settings, often in online forums, or synchronous internet relay chat16. Project blogs written by scientists can also provide a venue for interaction between the project scientists and the citizen scientists, and participants are able to comment on new posts17. The topics that are discussed can vary widely. For example, new participants can ask those who are more experienced questions relating to the project task or about the project more generally18. Participants may also use them as tools to interact more generally with each other.

Previous research has shown that participating in online communities and interacting with other participants can be an important motivator for some participants in online citizen science projects (Holohan and Garg, 2005, Raddick et al., 2013). Jennett et al.

(2013) refer to ‘sociability’ in online citizen science projects, and suggest that participants may be motivated to take part because of the social interaction with other participants, or with scientists. They state that this interaction may help foster feelings of community and that there is potential for many different types of sociability to develop given the right project platform, although how participants interact within online citizen science

16 Internet relay chat (also known as IRC) facilitates transfer of messages in the form of text. Applications are based on a client/server model of networking and are used to enable synchronous communication in a group setting as well as between two individuals. 17 The Galaxy Zoo Blog: http://blog.galaxyzoo.org/. 18 The GalaxyZoo forum has examples of this: http://www.galaxyzooforum.org/index.php?PHPSESSID=omd860f8o896cr5ev2jqof37r0&board=13.0. 40 projects has not been described or explored in much depth in other studies (Holohan and

Garg, 2005, Mugar et al., 2014).

Holohan and Garg (2005) explored interaction between participants within two distributed computing projects (SETI@home and the Great Internet Mersenne Prime

Search). Through surveys and interviews with participants, they found that social interaction with other participants was one of the most important aspects of involvement and a strong motivator for over half of respondents. They also claim that interactions were stronger within the various project teams as participants worked together to generate the most points and improve their ranking on the project leader board.

However, little detail or analysis was presented relating to these interactions.

Recent work by Mugar et al. (2014) has explored interaction between citizen scientists relating to learning the project task on two different online citizen science projects, Planet

Hunters and Seafloor Explorers. They state that in order to sustain groups in the long- term, newcomers to the group need to learn how to be effective participants. Their work details how new project participants learn about the tasks by examining ‘work in progress’ through the online discussion threads, and by asking more experienced participants for advice. This is known as ‘legitimate peripheral participation’, a term first used by Lave and Wenger in their work on ‘communities of practice’ (for example in skilled trades such as midwives, tailors or butchers) (Lave and Wenger, 1991).

This work describes the process of development from newcomer to an accomplished insider, as community members become more familiar with the tasks and practices of that community. In a community of practice, joint sense-making and problem solving enhances the formation of interpersonal ties (McLure Wasko and Faraj, 2005). Members, or practitioners, develop a shared repertoire of resources, experiences, tools and other 41 ways of addressing recurring problems (Wenger, 2006, Hanson-Smith, 2013). Over the past decade or so, further thinking about communities of practice has extended the meaning of the term to include online communities and interactions (Makriyanni and De

Liddo, 2010, Hanson-Smith, 2013). Such communities of practice may be of relevance to online citizen science projects, and could result through sustained interactions between participants.

While interaction between project participants may generate ‘sociability’ it may also form the basis for co-operation or collaboration between participants. Co-operation and collaboration are two distinct processes. The main difference is whether the tasks are divided up and individually completed (i.e. co-operation), or whether they are completed together through dialogue (i.e. collaboration) (Paulus, 2005). Co-operation involves a division of labour and possibly task specialisation, and individuals take responsibility for their part of the final product. In contrast, collaboration is a co-ordinated, synchronous activity, in which there is a shared concept of a problem and a process of shared creation

(Dillenbourg, 1999).

Much previous work on collaboration and co-operation has come from the literature on education, and has focussed on the role of collaboration and co-operation in learning both in ‘real’ and ‘virtual’ environments (Dillenbourg, 1999, Holliman and Scanlon, 2006,

Lai, 2011, Paulus, 2005). Online citizen science projects are occasionally referred to as

‘collaborations’ between citizen and professional scientist, although there is, as yet, little documented evidence exploring how (or if) participants in these projects collaborate, or co-operate to achieve the research aims of the project.

While there are many opportunities to interact with others taking part in an online citizen science project, there is little information relating to what proportion of participants take 42 part in these interactions. This research has explored interaction between participants in greater detail. For each of the selected projects in this research, data has been collected relating to who takes part in online interaction, how they interact and why they interact.

Interaction has been considered with regard to motivation to participate, and within the wider context of how participants contribute to online citizen science projects.

Opportunities for collaboration and co-operation have also been explored. The following research question focuses on this area.

Research question 4: How and why do project participants interact online?

Not only has interaction between citizen scientists been considered, but also the interaction between citizen scientists and those who manage the projects (scientists and developers.

2.6 Contribution to online citizen science projects

Initial observations of online citizen science projects has highlighted the fact that while some projects can have tens of thousands (or even hundreds of thousands) of registered participants, only a small proportion actively contribute to a project. And in some cases

(e.g. Stardust@home) the numbers of participants decreases rapidly over time (Nov et al.,

2011a). The concept of ‘registered’ and ‘active’ users, while alluded to in some previous work (Krebs, 2010, Luczak-Rösch et al., 2014), has not been explored in any detail. It is hoped that this research will shed some light on this phenomenon, and the numbers of active vs. registered participants for the three selected projects are estimated in Chapter

Four.

Observations have also demonstrated that there are other ways to get involved in a project in addition to the main project task. Some citizen scientists get involved in

43 moderating forum discussions, translating project content, helping new participants learn the project task, and writing content that can support a project such as FAQs or project wikis. Little work has been carried out that explores the different roles that may be available to project participants, and to what extent citizen scientists become involved in them.

A small number of studies exploring Zooniverse projects have considered different levels of contribution. A study of participants in Old Weather, a project that involves the transcription of archived Navy logs, refers to two different ‘types’ of participant: high contributors and low contributors, or ‘dabblers’ (Jennett et al., 2014). High contributors are more engaged by the social and competitive features of the project, but make up a small percentage of the overall number of participants. Most contribute on a much smaller scale, ‘dabbling’ in the project often for a short period of time. They are less likely to become involved in the some of the social features of the project.

Another study that examined the pattern of participation in a number of Zooniverse projects found that there was a community of highly active users who, in addition to making contributions through the completion of the project tasks, also made the most contributions to the online discussions, thus becoming a ‘core community’ (Tinati et al.,

2014). A similar finding was made by another group examining 10 Zooniverse projects

(Luczak-Rösch et al., 2014), and by a group looking at participation in Galaxy Zoo and the

Milky Way Project (Ponciano et al., 2014).

This pattern of ‘uneven’ contribution has been explored in detail in relation to other online communities. While hundreds of millions of people use the Internet, only a small fraction of them move from just reading content, to become contributors of user- generated content (Ciffolilli, 2003, Brake 2014). Some contributors move beyond this 44 individual effort and become collaborators and form connected networks with others with a particular focus (e.g. a Wikipedia article, or an online game wiki). Of this group of collaborators, an even smaller number of participants may become involved in activities such as helping novices, or establishing and enforcing community policies (Kittur et al.,

2007, Brandtzaeg and Heim, 2009, Makriyanni and De Liddo, 2010).

Preece and Schneiderman (2009) have described this pattern of participation in more detail in their ‘reader-to leader’ framework (Figure 2.1), and it describes the journey that some individuals make from reading content, to contributing content, to collaborating with others, and eventually becoming a ‘leader’ of the community. While the number of

‘readers’ may be great, the number of individuals moving to each successive stage rapidly decreases (Preece and Shneiderman, 2009).

Whether this pattern of participation can be observed in online citizen science projects has not been considered in detail, and a further exploration of this framework may help to illuminate the nature of contribution in online citizen science projects. This framework goes beyond a quantitative description of how much a participant contributes, and considers the types and nature of the tasks at each stage.

Some online citizen science projects may eventually result in the establishment of an online community of practice perhaps consisting of a small highly-motivated group of

‘collaborators’ and / or ‘leaders’. In such groups, participants would be expected to become more involved and adept at the project task, or perhaps become involved in other project-related tasks such as moderating forums, teaching new participants, developing a project wiki resource, or directly providing feedback and recommendations to the project scientists. An exploration of this framework in online citizen science projects may also be useful in a consideration of online interaction. For example, there 45 may be little interaction between ‘contributors’, but more substantial interaction between ‘collaborators’ or ‘leaders’.

Figure 2.1: the ‘reader-to-leader’ framework (Preece and Schneiderman, 2009)

An alternative framework for exploring patterns of contribution in online citizen science projects can be found in the work of Haythornthwaite (2009) who has examined peer production communities in detail, and describes contributory behaviours as either

‘lightweight’ or ‘heavyweight’, and carried out by either ‘crowds’ or ‘communities’. In lightweight peer production, individuals can easily contribute, and there is usually a large set of participants (the crowd) who provide minimal additions to the endeavour as a whole. The ‘rules’ of contribution are defined by authorities or owners of such projects, and participants are not expected to play a role in determining the direction or the project as a whole. Participants do not need to make long-term contributions, nor do they need to interact with others. They are free to dip in and out when time or inclination allows. Haythornthwaite uses distributed computing projects as an example of lightweight peer production.

In heavyweight peer production success depends upon a critical mass of contributors (the community) who make significant time investments to the project and who interact with other participants in order to sustain the community. There are learned norms of interaction and language which are indicative of community membership. Outsiders or novices can be easily identified. In some cases, the participants determine the goals of the project. Haythornthwaite uses the academic community as an example of heavyweight peer production.

While some endeavours can clearly be defined as either lightweight or heavyweight, there are some examples where there is some overlap. For example, Wikipedia demonstrates both lightweight behaviour from the crowds who edit and update articles, and heavyweight behaviour from the small community of editors who act as gatekeepers and decide which articles remain. Table 2.3 outlines the main features of lightweight and heavyweight peer production.

This framework appears to be relevant to online citizen science as it encompasses a range of tasks and behaviours, some of which are readily observed in a number of projects. It also affords some flexibility as projects may exhibit characteristics of both lightweight and heavyweight behaviours, which may be relevant to specific project tasks.

This thesis explores contribution from a quantitative perspective including the identification of active participants and a consideration of the level of contribution (e.g. how many hours a week do individuals devote to a project). It also explores the diversity of roles and tasks that are available to participants in each of the three selected projects and to understand what citizen scientists do while they are participating.

Table 2.3 Main features of lightweight and heavyweight peer production (Haythornthwaite, 2009)

LIGHT HEAVY Contribution type, granularity and authentication Atomistic, independent Connected, revised, negotiated Addressing uncertainty, explicit knowledge Addressing equivocality, tacit knowledge Rule-based contribution Negotiated contribution Delimited contribution attributes Variable contribution attributes Single form defined by authority / owner Multiple forms defined by and authenticated by group consensus, norms Pooled interdependence Reciprocal interdependence Individual to group focus Anonymous Attributed History of contribution unnecessary History of contribution important for group Open membership, low threshold to entry Review, gatekeeping to join, high effort for membership Two-tier hierarchy: authority and contributor Multi-tier hierarchy: novice to expert, newbie to experienced Independent, repetitive , discrete contributions Continuing, contingent, norms-based contribution to product and process Recognition, reputation, reward Quantitative recognition mechanism Qualitative recognition Internally relevant to the individual application or the Internally relevant, permeable to field of interest arena of contribution Quantitative measures of contribution to product Internal: judgements of contribution quality, expertise re field of interest Peer review judgements of contribution to products and process

The ‘reader-to-leader’ framework, and the ‘lightweight-heavyweight’ framework for peer production will be used to consider the level and types of contribution observed in the three selected projects, and will be explored in the analysis of the findings in Chapter

Eight. Thus, the following research question will be addressed.

Research Question 5: How can contribution to online citizen science projects be characterised?

A further aspect of participation, which has been little explored, is how participants (both citizen and professional scientists) view their contribution and involvement in online citizen science projects. For example, do they feel they are actively involved in scientific research? The application of the ‘reader-to-leader’ framework may provide some insight

48 here also, and participants may view themselves as contributors, collaborators or leaders

(or perhaps, none of these things). The final research question explores this area in more detail.

Research Question 6: How do participants perceive their role in the project?

A greater insight into the experiences of participants in this respect may also increase our understanding of how these projects are sustained over time, and why individuals participate.

2.7 Conclusion

Online citizen science projects have increased steadily in number over the past decade

(2004-2014). Hundreds of thousands of individuals have participated in these projects, and new projects continue to appear. Furthermore, there have been significant scientific developments as a result of the efforts of citizen scientists and these have been published in a number of high-profile science journals (Khatib et al., 2011a, Khatib et al., 2011b,

Lintott et al., 2009, Lintott et al., 2008, Schwamb et al., 2013, Wang et al., 2013, Schmitt et al., 2014).

This analysis of the literature has identified a number of gaps in the understanding of online citizen science projects, particularly relating to who participates, why they participate and how they participate. While previous research may have explored a single aspect of participation (most notably motivation to participate), there has been less consideration with regard to how these aspects of participation may be inter-related.

While the research questions aim to address all of these aspects of participation, the subsequent analysis will aim to adopt a more holistic approach to understanding online citizen science projects, and explore how these aspects of participation may be connected. This research will also examine how these aspects of participation vary 49 between different types of online citizen science project, as there has been very little comparative work carried out in this area so far (Nov et al., 2011b).

The following research questions will be addressed.

1. Who is participating in online citizen science projects?

2. What motivations initiate and sustain participation in online citizen science projects?

3. Do motivations vary between different types of online citizen science projects and their associated tasks?

4. How and why do project participants interact online?

5. How can contribution to online citizen science projects be characterised?

6. How do participants perceive their role in the project?

Three separate projects have been investigated which have different types of tasks, and different online platforms. One project from each of the major ‘types’ of online citizen science project (See Chapter One, Table 1.1) have been selected: Folding@home (a distributed computing project); Planet Hunters (a distributed thinking project); and Foldit

(a citizen science game). These projects will be described in greater detail in Chapter

Four. While I have examined three specific cases, some of the findings may be relevant to other online citizen science projects, and may provide some practical information for those considering setting up or managing a project of their own.

Chapter 3: Methodology and Methods This chapter will outline the rationale for the overall research design. I will discuss general methodological approaches that have been considered, and the research methods chosen for data collection and data analysis.

3.1 Methodological approach

This is an empirical study that is directed towards addressing a set of research questions that have arisen after an interrogation of the literature. The data collection and analysis are subsequently directed towards the elucidation of these research issues. This research has not been directed by any one theory, although one of the aims of this work is to explore and develop concepts and frameworks which may be used to explain some of my observations relating to online citizen science projects.

3.1.1 A mixed methods approach

In order to address the research questions, attitudes and opinions were sought from those who participate in online citizen science projects. To explore these fully and to give participants greater freedom in their responses, a mainly qualitative approach to data collection and analysis was undertaken. With the exception of the study of Raddick et al.

(2010) where 22 Galaxy Zoo participants were interviewed, such an approach has not been taken in previous studies of online citizen science. Much of the previous work has used multiple choice, Likert scale-based surveys to explore participant motivations (see

Chapter Two, Table 2.3). Such an approach can be too prescriptive and answers or responses ‘outside’ the box may not be communicated (Jamieson, 2004, Carifo and Perla,

2007). However, in order to understand some of the parameters of participation (such as how much time an individual commits to a project and the different types of participation) as well as some of the characteristics of the participants (such as

51 demographic characteristics and participation in related activities), quantitative data was also collected.

The combination of different approaches to data collection and analysis is known as a mixed methods approach, and can be described as a class of research where the researcher mixes or combines research techniques, methods, approaches, concepts or language into a single study (Bryman, 2006, Cresswell, 2009). A mixed methods study usually employs a mixture of qualitative and quantitative methods and enables research questions to be answered using a variety of methods such as quantitative surveys, qualitative surveys, interviews, systematic observation, or visual data to tap into different aspects of behaviour (Symonds and Gorard, 2010).

A mixed methods approach also introduces potential for methodological triangulation, in which inferences or results from each method are used to confirm, corroborate or confound each other, thus providing a strategy for ‘cross-checking’ (Bryman, 2012,

Maxwell, 1996, Symonds and Gorard, 2010). Triangulation can help to reduce the particular limitations that are associated with any one method, and can make a dataset more robust (Jensen and Holliman, 2009). In addition to methodological triangulation, there is investigator triangulation which involves cross-checking interpretations between researchers; and data triangulation where data from a variety of sources is brought together to explore similar issues (Carr et al., 2009). All three types of triangulation will be incorporated into this study through the adoption of a mixed methods approach.

Both qualitative and quantitative methods have their own strengths and weaknesses.

Quantitative research provides numerical data that is useful for testing and validating already constructed theories, and which may be generalized to a wider population under certain conditions (Bryman, 2012, Sapsford, 2007). One potential problem with 52 quantitative research may be that the researcher’s categories, or indeed their theories, may not reflect the understandings or experiences of those who may be the subject of the research (Silverman, 2006).

Qualitative research is useful for describing complex phenomena, and the data are based upon the participants’ own ‘category of meaning’ (Taylor and Boyden, 1998), although this too is subject to the interpretation of the researcher(s). They are often collected in naturalistic settings, and can provide an understanding or description of people’s personal viewpoint of a phenomenon (Maxwell, 1996). Individual case information may be obtained, as can cross-case comparisons and analyses. Knowledge generated by qualitative research may be difficult to generalise to other settings or populations, and it can be more difficult to test theories and hypotheses (Thomas, 2009).

Some researchers argue that a mixed methods approach allows the researcher to offset the weakness of both quantitative and qualitative research and draw on the strengths of both, and there is an underlying recognition that both qualitative and quantitative research are important and useful (Symonds and Gorard, 2010). This approach also attempts to bridge the different epistemological beliefs that are often associated with a strictly quantitative (positivist) approach, or a strictly qualitative (relativistic or constructivist) approach (Cresswell, 2009).

For example, Johnson and Onwuegbuzie (2004) advocate a needs-based approach to research method selection, and suggest that a paradigm based on pragmatism best describes the mixed methods approach. A pragmatic approach rejects dualisms (e.g. subjectivism vs. objectivism) in preference for more moderate versions based on how well they could work in solving research problems. Pragmatism views truth as changing over time, and views knowledge as being both constructed and based on the reality of the 53 world that we experience and live in (Johnson and Onwuegbuzie, 2004). Such an approach sits well with my own research background which is the quantitative biological sciences, with some experience in researching the wider area of ‘science and society’

(Curtis, 2013).

One important consideration when it comes to conducting any type of research is the role of researcher bias. This occurs when the personal feelings and values of the researcher influence the way research is both carried out and subsequently analysed (Thomas,

2011a). It is relevant for both quantitative and qualitative data, and there are numerous points at which the researcher’s personal biases and idiosyncrasies can be introduced including:

 choice of research area;

 formulation of research questions;

 choice of method;

 formulation of research design;

 analysis of data;

 interpretation of data;

 conclusions.

Some state that the results of quantitative research may be less prone to researcher bias, where data collection and analysis are influenced by the researcher’s beliefs, background and idiosyncrasies (Bryman, 2012, Johnson and Onwuegbuzie, 2004). However, others dispute this stating that researchers using mainly quantitative techniques are by no means exempt from this bias, and that their beliefs and backgrounds also directly

54 influence what they choose to research, the methodologies they employ and how their analyse their data (Irwin and Wynne, 1996).

My own approach will have undoubtedly been influenced by my background and beliefs.

For example, my background interest in science and my belief that scientific research should be accessible has strongly influenced my choice of online citizen science projects as the subject of my study. My academic background in science will also have influenced how I selected the projects for my research, and I have chosen projects that are in areas of science that are of interest to me. The challenge for the researcher is to be aware of these potential sources of bias, to consider them at each stage of the research process, and to ask oneself if they are adversely affecting the quality of the research.

Shenton (2004) states that triangulation of data can help to address the problem of researcher bias in qualitative research in that it can help to define a study’s

‘confirmability’. This is a concept related to objectivity, and involves steps to ensure, as far as possible, that the work’s findings are the result of the ideas and experiences of the subjects, rather than the characteristics and preferences of the researcher (Shenton,

2004).

3.1.2 Case studies as a mixed-methods approach

Case study research is an approach that can be used for conducting research in both the natural sciences and the social sciences (Yin, 2003, Thomas, 2011b). The case study is a focus on one thing: an individual; an organisation; an event; a project, which can be examined in depth and from many angles. Gillham (2000) describes a case study as a unit of human activity embedded in the real world which can only be studied in context. Case studies can be utilised to gain a rich picture of real-life circumstances in order to obtain analytical insights (Baxter and Jack, 2008, Yin, 2003). A case study approach has not been 55 adopted in previous studies on online citizen science, and thus, a depth of focus has been largely missing from the body of this work.

Multiple sources of evidence and lines of enquiry are employed to examine a case in great detail, and this ‘chain of evidence’ can be used to address research questions (Yin, 2003).

The types of evidence that can be used in a case study are numerous and wide-ranging.

They can include interviews, personal accounts, diaries, group interviews, focus groups, archives, statistics, questionnaires, participant observation, and physical artefacts

(Thomas, 2011a). Thus, case studies lend themselves to a mixed methods approach, where both qualitative data and quantitative data can be utilised, and where triangulation of data may be facilitated.

Case study research can be used to both test and develop theories. Studies can focus upon a single case, or a number of cases may be studied jointly in order to investigate a phenomenon, or to form the basis of a comparative study (Yin, 2003). They can be prospective, retrospective, sequential or parallel, in that events occurring in one case study, may affect the events in a subsequent case. Indeed, a number of typologies have been developed to describe case study research, with no one universally accepted approach (Baxter and Jack, 2008, Gerring, 2007, Gillham, 2000, Thomas, 2011b, Yin, 2003,

Zucker, 2009).

Instead of a rigid typology, Thomas proposes a set of descriptors to be used in relation to various aspects of the case study, including the subject, the purpose, the approach and the process (Thomas, 2011a). This may allow for more flexibility in describing different case study approaches and enable a more detailed consideration of its various components (see Figure 3.1). Thomas’ approach will be used to describe my own online citizen science case studies in the next section. 56

Figure 3.1 Descriptors of case studies (from Thomas, 2011a, p.93)

Subject Purpose Approach Process Special or outlier case Key case Local knowledge case Intrinsic Instrumental Evaluative Explanatory Exploratory Testing a theory Building a theory Illustrative Descriptive Interpretive Experimental Nested Parallel Single or multiple Sequential Retrospective Snapshot Diachronic

There is some discussion surrounding the use of case studies and much of this has centred around the generalisability of the findings, and whether they can be used to provide a basis for drawing conclusions about related types of phenomenon or about members of the wider population of cases (Gomm et al., 2000, Hammersley and Gomm,

2000). However, proponents of case study research have argued that generalisation is not always what is wanted in the research process, and a case study is about seeing something in detail (Thomas, 2011b, Yin, 2003, Flyvbjerg, 2006, Thomas, 2011a) . Others state that case studies may permit comparison. For example, Goetz and Le Compte

(1984) outline the concept of ‘comparability’, which they define as the degree to which the components of the case study, including the units of analysis, concepts generated, population characteristics and settings, are sufficiently well described and defined so that other researchers can use the results of the study as a basis for comparison. This issue of

57 comparability will be explored in this research, as I investigate how the three online citizen science projects selected can be compared and contrasted.

Some have argued that reproducibility is not a principle concern in case study research, as there can be no assumption from the outset that if the study were to be repeated by others at a different time, then similar findings would result (Schofield, 2007, Thomas,

2011a). Thomas (2011a) states, that while normative measures of validity may not be as applicable to case study research as it is to other types, the quality of the research is important. Quality and rigour in a case study can be assessed by how well the cases have been chosen, decisions taken relating to data collection and analysis, development of arguments and theories, exploration of rival explanations, ethical conduct, and the overall clarity of writing (Thomas, 2011a).

Some claim that case studies are more susceptible to researcher bias towards verification, that is, a tendency to confirm the researcher’s pre-conceived notions (Flyvbjerg, 2006).

However, proponents of case study research see this criticism as a reflection of the lack of knowledge about the methodology, although Yin emphasises that the researcher must constantly be aware of rival explanations (Yin, 2003), while Gillham reminds the researcher to be mindful of their prejudices during research (e.g. what do I think I will find?) and of their preferences (e.g. what do I want to find?) (Gillham, 2000).

3.1.3 Online citizen science projects as case studies

As a case study approach lends itself both to an in-depth understanding of a phenomenon as well as a comparison between cases, it has been selected to address my research questions. A case study accommodates a mixed methods approach to data collection and analysis, and they are also useful when a subject is being encountered for the first time, or being considered in a fundamentally new way (Gerring, 2007). Chapter Two has 58 illustrated the extent to which research has been published on online citizen science.

Although this body of research is growing, what has been undertaken so far has had a relatively narrow focus (e.g. motivation). An in-depth consideration of the three projects permitted by a case study approach will contribute to our understanding of online citizen science by producing detailed findings built upon multiple sources of evidence.

Referring back to Figure 3.1 and the descriptive indicators of case studies outlined by

Thomas (2011a), it is possible to describe my approach to each of the cases selected for this research using this approach as a guide (see Figure 3.2).

Figure 3.2 Mapping out the design for the online citizen science case studies

Subject Purpose Approach Process Key case Exploratory Explanatory Descriptive interpretive Multiple cases Snapshot

The subject of each case study could be considered a ‘key case’ as each one represents a distinct type of online citizen science project. The purpose of each case study is both to explain and explore the patterns of participation in these particular projects, while the approach involves both a description of the case and an interpretation of the findings that will address the research questions. Finally, the research process will involve the comparison of multiple case studies and provide a ‘snapshot’ of that particular project

(i.e. they will not be longitudinal studies). Each of the three cases has been approached in the same way with regard to data collection and analysis. This will enable a comparison to be made of the cases with greater accuracy.

3.2 The online research setting and ethical considerations

After getting permission from the first of the project managers (Foldit) to carry out my research, I sought permission from the Open University Human Research Ethics

Committee (HREC) and was granted permission for my research involving other project participants on the basis that any interaction would be restricted to those aged 16 and older. The Foldit online survey was launched shortly after.

Research involving online settings is increasing in prevalence, and ethics boards may have to reconsider issues such as consent, risk, privacy, anonymity, and confidentiality

(Buchanan and Hvizdak, 2009). Use of the Internet is continuing to evolve, and the methods used to explore it are emerging and developing (Eynon et al., 2008). This requires a regular reappraisal of ethical issues, and researchers in this area have to keep abreast of evolving ethical guidelines.

One of the key challenges in this research has been the concept of private vs public space.

Indeed, much debate about internet research has centred on this topic (Kozinets, 2010).

Project websites are in the public domain, and serve as the public interface and route of access for a project. These websites also have online discussion forums where participants can interact and discuss issues relating to the project, or indeed, issues that are totally unrelated. While these discussions are also easily viewed and in the ‘public domain’, one needs to consider whether participants would be happy for their discussions to be the subject of research, and also what the ‘terms and conditions’ are for each project (Kozinets, 2010).

The concept of privacy has been considered to varying degrees in the projects. Foldit has terms of ‘service and consent’19 where some of these considerations are communicated.

For example, participants are reminded that: “The website and chat can be recorded.

Other people can see what you post.” Projects that are on the Zooniverse platform, which includes Planet Hunters, share the same ‘user agreement’20. While the custodians of this project agree to not share email addresses, they do state that: “Specifically, we may share your anonymized data with research study participants, other researchers, or in scholarly work describing our research.” They also add that “Contributions you make to the Talk pages are widely available to others.” Folding@home has a privacy policy21, but this is related to the data utilised by the project scientists, and there is no mention of visibility of contributions to any online discussions. Planet Hunters (via The Zooniverse) appears to be the only project that explicitly mentions research undertaken by third parties.

However, there is no way of ascertaining how many project participants read these terms and conditions, or to what extent they understand the implications.

There appears to be little research examining the views of those who take part in online communities regarding research by third parties (Bruckman, 2006). However, one study of online internet chat rooms has found that some participants are openly hostile to the presence of researchers (Hudson and Bruckman, 2004). While a chat room is accessible to the ‘public’, participants may feel like they are part of a trusted community and like to feel able to express themselves without feeling that they are being ‘watched’ or that their words are being analysed (Eynon et al., 2008), and this must be taken into consideration when exploring online spaces including chat rooms, online forums and discussion boards.

19 The Foldit terms of service and consent (accessed 18/02/14) http://fold.it/portal/legal. 20The Zooniverse user agreement (accessed 18/02/14) https://www.zooniverse.org/privacy. 21Folding@home policies, including a privacy policy http://folding.stanford.edu/home/faq/faq-policies. 61

There are a few guidelines for researchers working with online projects and communities

(Bruckman, 2002, Markham and Buchanan, 2012, British Psychological Society, 2007). I have used the guidelines put together by Bruckman (2002) when approaching the use

(quoting and analysing) of material in the ‘public domain’, which are as follows:

1. You may freely quote or analyse online information without consent if:

- it is officially, publicly archived

- No password is required for archive access

- No site policy prohibits it

- The topic is not highly sensitive

2. For everything else not covered by 1, you typically need consent. (Bruckman, 2002,

pp 1)

In order to address some of these wider ethical issues I tried to make sure that other project participants were aware of my presence on the project, and that the nature and objectives of my research were clearly presented. This was achieved through postings on online forums where I discussed my research, or through project blogs, where project scientists mentioned my research22. It was also mentioned on the online survey form

(Appendix A). I was an active participant in each project and took part in the projects and online discussions to varying degrees (see Section 3.3.2 and Chapter Four).

3.3 Methods

In addition to ethical considerations relating to carrying out research online, there are also methodological considerations relating to using online research tools such as online surveys, and carrying out online interviewing. These will be considered as I outline the main methods employed as part of this research.

22 Link to discussion thread about my research on the Foldit forum: http://fold.it/portal/node/993215. 62

3.3.1 The selection of the cases

One project from each of the three types of online citizen science project was chosen for this research. Their selection was based on a number of criteria (Table 3.1) which would best address the research questions and add to the diversity of projects that have been previously been explored.

Table 3.1 Selection criteria for cases

1. Project should not have been the focus of any other study at the time of its selection.

2. Projects should ideally be at least year old and relatively ‘established’.

3. Project should have an online community.

4. Project scientists / developers should be ‘visible’ and interact to some degree with participants.

5. Research objectives are made clear.

As part of the selection process, I also tried many of the projects myself, and this provided an efficient means for making an assessment based upon the above criteria. In total, I considered three distributed computing projects, three distributed thinking projects and two citizen science games. These are listed in Table 3.2.

Following these preliminary investigations, an initial selection of my three first-choice projects was made. For each of these, the scientist(s) in charge of setting up and managing the project were contacted via email. My research aims and intentions were outlined, and I asked for permission to have access to the project and to make contact with project participants. All three of these project managers agreed to grant me access to their projects.

Table 3.2 Projects considered as case studies

Distributed computing Folding@home Simulates process of protein folding. projects LHC@home Simulates sub-atomic particle collisions. Einstein@home Searches for evidence of pulsars. Distributed thinking projects Planet Hunters Search for planets outside of the solar system. Whale FM Groups individual whales based on recordings of their song. Seafloor Classification of items on a section of the Atlantic ocean Explorer floor. Citizen science games Foldit Protein folding game. EteRNA Game based on the synthesis of RNA molecules.

Table 3.3 lists the projects finally selected and when project managers were contacted. It also outlines the general research aims of each project, when and where it was established, and an estimate of the number of registered participants. Each project will be described in more detail in Chapter Four, and the number of ‘active’ participants in each project will be discussed.

Foldit was the first case study to be initiated, followed by Folding@home and Planet

Hunters. However, there was some overlap with the collection of data between all three projects. All of the data was collected between June 2012 and February 2014.

Table 3.3 projects selected for case studies

Project Type of project & general When / where Number of research aims established registered participants (as of 2014) Foldit Citizen science game 2008 Over 500 000 (www.fold.it) Participants aim to deduce University of registered players Contacted March 2012 the 3-D structure of Washington, listed on website. protein molecules through Biochemistry Dept. a stylised multi-player and Centre for online game. Games Research, (USA) Folding@home Distributed computing 2000 Estimates vary (http://folding.stanford.edu/) Participants run protein Stanford University between 27 000 Contacted Feb 2013 folding simulations on Chemistry Dept. and 100 000. computers or games (USA) consoles. Planet Hunters Distributed thinking 2010 Approx. 160 000 (www.planethunters.org) Participants look for Oxford University Contacted Feb 2013 evidence of exoplanets in (UK), Yale data from NASA Kepler University and mission Adler Planetarium (USA)

3.3.2 Streams of evidence to be considered

In case study research a number of different ‘data streams’ may be utilised in order to gain an in-depth understanding of that case. These can include quantitative surveys, interview data, participant observer data, user logs, archive material, external publications (e.g. media articles, web content) and audio-visual material. For each online citizen science project I have selected three different data sources to address the research questions. While some information is already available on the project websites and from external publications there is not enough that addresses my areas of interest and so data has been generated through questionnaires and interviews.

The following sources of data have been considered for all three cases and incorporated into the research analysis.

1. Information and experiences gathered through being a participant-observer in

each of these projects;

2. Online surveys of citizen scientists taking part in each project;

3. Qualitative, semi-structured interviews of a sub-set of citizen scientists who

completed the survey, and of the professional scientists and developers involved

in managing the projects.

The timeline for data collection / generation for each of the projects is illustrated in Table

3.4.

Table 3.4 Timing of data collection and generation

Project Date I joined Online Follow-up Follow-up interviews of project as a question interviews of project scientists participant naire citizen launch scientists

Foldit April 2012 June Jan – Feb 2013 June 2012 - Feb 2014 2012 Folding@home Feb 2012 March Jan – Feb 2014 Feb 2014 2013 Planet Hunters Jan 2011 23 May Nov 2013 – Jan Jan-Feb 2014 2013 2014

Data was collected over a number of months, and the approach I have adopted can be described as ‘exploratory sequential’ meaning that there is a sequence of data collection, with the results obtained through one method informing the approach or content of another (Guest, 2013). For example, online survey questions have been informed by

23 I initially took part in Planet Hunters shortly after its launch when it was highlighted during the 2011 BBC Stargazing Live programme. This is a programme designed to promote astronomy and often features amateur astronomy and citizen science activities. While this involvement only lasted a few weeks, I became a more active participant in February 2012. 66 participant observation, while the interview schedule has been influenced by the questionnaire data and my experience as a participant observer. The process of conducting surveys and interviews has influenced my experience as a participant observer. Figure 3.3 illustrates how the data collection processes have been interconnected.

Figure 3.3: Data collection process (‘exploratory sequential’)

Participant-observer role

I have been a participant observer in all three case studies. In one of the projects, Planet

Hunters, I began participating prior to this research (See Table 3.4) and had become familiar with some of the other Zooniverse projects as part of an MSc dissertation on astronomy outreach (Curtis, 2013). Being a participant observer is a direct way of obtaining data by observing what people actually do (Gillham, 2000). It can be used as an 67 exploratory technique, as a supplementary or additional source of information (as in this research) or it can form the basis of a research project resulting in the production of a detailed ethnography (Hammersley and Atkinson, 2007).

Participant observation has a number of advantages, namely, that it gives the researcher access to the ‘background culture’ and can enable a detailed description of behaviours, situations, intentions and events that they may not have access to otherwise (Kawulich,

2005). However, there can be problems with this research tool. For example, the researcher may be reliant on a small number of informants, and the experiences and views of these individuals may not be typical of the community in question (Kawulich,

2005). Perhaps one of the most important considerations is researcher bias. The researcher must be aware that their gender, ethnicity, class and theoretical approach may affect their observation and interpretation (Iacono et al., 2009).

The degree to which a researcher becomes integrated into the community can vary greatly, and some researchers talk of a ‘continuum of participation’ (DeWalt and DeWalt,

2002) or ‘boundaries of insiderness’ (Labaree, 2002). Determining the degree of participation and membership of the group is sometimes done by the researcher, and sometimes it is decided by the community (DeWalt and DeWalt, 2002). The fact that the communities that I was observing were online also influenced the degree to which I could become involved. For example, much of the communication with other participants was asynchronous and conducted through online forums or discussion boards. This alters the temporal nature of exchanges, and can create distance between participants (James and

Busher, 2009a). Interactions that took place during real-time synchronous internet relay chat felt more spontaneous and immediate. The project tasks also influenced the degree to which I could enter the project community. For example, one project task (Foldit) was

68 quite complex and necessitated much more interaction with other participants (in order to learn how to complete the task) than in the other two projects.

The degree to which I achieved membership varied. Table 3.5 summarises the

‘continuum of participation’ as proposed by DeWalt and DeWalt (2002) and I have marked where my involvement with each of the projects is situated in this continuum.

Table 3.5 The continuum of participation (from DeWalt & DeWalt, 2002)

Non-participation No membership role. Observation only. Researcher remains external to the community

Passive participation No membership role. Researcher is on the spot, but acts as an observer (as a spectator or bystander)

Moderate participation Peripheral membership. (Folding@home) Researcher is present at the scene of action but does not participate, or only occasionally interacts with people.

Active participation Active membership. (Planet Hunters) Researcher engages in almost everything that other people are doing.

Complete participation Full membership. (Foldit) Researcher is, or becomes a member of that group.

This approach describes the various roles that the participant-observer may take. The roles range in degree from non-participation (activities are observed from the outside), to passive participation (activities are observed in the research setting but without participation in any of the activities), to moderate participation (activities are observed in the research setting with occasional participation on activities), to active participation

(activities observed in the setting with almost complete participation in activities), to complete participation (there is complete participation in the ‘culture’). While the degree of participation varied between projects, this did not affect my ability to collect data or to make observations of other participants.

For each project a similar participant-observer protocol was adhered to (for Planet

Hunters this protocol was followed as soon as I received permission from the project managers to carry out my research in February 2013). This is summarised in Table 3.6.

Table 3.6 Participant Observer Protocol

1. Register with project.

2. Download software, complete tutorials on project task.

3. Register (when required) with online forum, and become familiar with project etiquette (e.g. online codes of conduct, presence of moderators).

4. Observe content and frequency of interactions between participants on online forum, internet relay chat (if available), and on project blogs. Identify key participants.

5. Attempt to estimate active participants in the project.

6. Take part in online forum discussions and internet relay chat when relevant or of interest (state research interests and affiliations on all first forum postings).

7. Explore other online project content produced by either participants or professional scientists (e.g. wikis, FAQs).

8. Participate regularly (4-5 times a week on average).

9. Write summaries of observations, reflections and of any interactions in project diary during active participation.

10. Compile a library of ‘screen shots’ to illustrate the online setting. These will supplement notes, and provide graphical illustrations of the areas where participants interact.

11. Share research findings with participants.

Acting as a participant-observer was the first stage in data collection. It enabled an exploration of the individual projects and provided an insight into how the projects were organised (see Chapter Four). This experience informed both the content of the surveys and the interview schedule, as well as providing a context for these findings during the subsequent analysis. Participant observation continued throughout the period of the research, although it was at a reduced level during the final stages of analysis and writing.

By continuing to participate, I retained my familiarity with the projects, kept abreast of new developments, and maintained contact with some individual participants. The 70 participant-observer protocol was formulated at the beginning of the research period with the exception of steps five and nine which were later considerations. Step five became important once it became clear that the number of active participants was significantly smaller than the number of registered participants. Step ten was introduced as the importance of screen shots in illustrating participant interaction began to emerge.

Online surveys of project participants

Online surveys can help a researcher reach a potentially large number of participants at relatively little expense. They can be set up quickly, and there are a number of online tools available to facilitate their development and administration (Kozinets, 2010). These tools also enable results to be effectively collated, and some enable basic statistical analyses to be undertaken (Sapsford, 2007). Many packages allow a significant degree of flexibility in layout and type of question that help the researcher tailor the questionnaire design to their specific research questions.

In addition to addressing issues relating to motivation and interaction, the online surveys were also used as an opportunity to collect demographic information about the participants which may have a bearing on the topics under consideration (particularly motivation to participate or how they perceived their contribution to the project). The results of the online surveys were later used to inform the direction and content of the interviews with both scientists and citizen scientists.

The design of a questionnaire must be carefully considered (Meadows, 2003, Tourangeau et al., 2000). A survey should not be too long, and it should be clearly set out and easy for the respondent to understand (Adams and Cox, 2008). I was also aware of the options relating to style of questions and did consider the use of Likert scales instead of more open-ended questions. Likert scales have been used in some previous work that explores 71 motivation to participate in online citizen science, and can enable a more quantitative analysis of results (Nov et al., 2011, Raddick et al., 2013). However, there was some concern that they may be too prescriptive. There is the possibility that if the researcher is not inclusive enough in the options / statements presented to respondents that something that was not previously considered may not be brought to light (Carifo and

Perla, 2007, Jamieson, 2004, Pell, 2005). Open-ended questions give respondents greater freedom to express themselves, although the responses may well be more time- consuming to analyse.

A questionnaire was designed that contained a mixture of closed questions and open- ended, opinion-based questions. The closed questions related to demographic characteristics such as age, sex, level of education, and indicators of their involvement, such as how long they had been with the project, and how many hours per week they spent on the project. The open-ended questions related to their opinions and attitudes.

For example, why they took part in the project, what they liked best about the project, things they would change, and how they would describe their interactions with other players. The design of the questions was informed by my participant-observation of each project, as well as previous research, and my own research questions.

Some questions were specific to certain projects. For example, the Foldit questionnaire asked participants whether they played any other computer games, as participants are regularly portrayed as ‘gamers’ in external media articles (Bourzac, 2008, McGonigal,

2012, Gross, 2012). In all instances, I offered the project scientists the opportunity to suggest one or more questions for inclusion in the survey. As they were permitting me access to their project, it seemed appropriate to offer them the chance to explore something that may be of interest to them. The Folding@home team were the only ones

72 to suggest a question, and they asked participants if they had had any technical difficulties with the latest version of the project software. Each questionnaire had an introductory page that outlined who I was, my institutional affiliation and the objectives of the research (Appendix A). The secure storage of data and the fact that data would not be passed on to third parties was clearly stated, as was the stipulation that only those aged 16 and over should complete the survey. This introductory section stated that I would share my research findings with project participants, as well as with the project scientists and the wider academic community24. A copy of each questionnaire can be seen in Appendices B-D.

A number of tools are available for constructing and carrying out online surveys. I used the Bristol Online Surveys (BOS)25 tool as this is widely used by the Open University to carry out its own student satisfaction surveys. All Open University postgraduate researchers are covered by a site license and there are personnel available to provide guidance and support. Once a final version of the survey was constructed, a test version was sent to each project team, who tested it with their own responses, and then approved it. A hyperlink to each survey was produced, and made visible to project participants via a number of channels. Table 3.7 illustrates where the survey links were posted for each project, and the final number of respondents.

24 Findings of the online surveys have been shared with key contacts in each of the project teams. A summary of the survey findings has been prepared for participants in all 3 projects. This has been shared by the project manager at Folding@home, and with those who took part in the Foldit survey. The Planet Hunters managers have stated that they intend to post a link to my findings on a blog post sometime in the near future. 25BOS homepage: http://www.survey.bris.ac.uk/. 73

Table 3.7 How participant surveys were advertised and final numbers of respondents

Project Where survey link was placed Final number of respondents

Foldit Online forum on website. The ‘what’s new’ section that is visible when game 37 is launched. Folding@home Online forum on website. Project blog. 407 Folding@home Twitter feed and Facebook page. Planet Hunters Online discussion boards. Email to participants. 118 Planet Hunters Twitter feed and Facebook page.

One of the main problems with online surveys is that the response rate can sometimes be poor (O'Brien and Toms, 2010). The number of respondents for the three surveys is relatively small in comparison to the overall number of registered participants, so these groups of respondents may not be representative of the entire population (see Table 3.3).

However, the small sample sizes may also be related to the fact only a relatively small proportion of the total registered users are contributing on a regular basis, and these sample sizes may therefore constitute a greater proportion of active participants. The fact that survey respondents were recruited through project forums and discussion boards also supports the likelihood that these individuals were actively making contributions to the project at the time of the surveys. Ascertaining the proportion of active participants is an important component of understanding who participates in these projects (RQ1), and an estimate of this figure will help to characterise the types of contribution (RQ5). A greater of understanding of registered vs. active participants will also help to place these samples sizes into the appropriate context and estimates of the active population for each project are presented in Chapter Four.

Those who participated in the survey are a self-selected sample as some individuals are more likely to respond to a questionnaire than others, or will have a greater interest in its

74 subject(s) (Sterba and Foster, 2008, Tourangeau et al., 2000). This survey could also be described as an example of ‘convenience sampling’, as it is composed of individuals who were available and chose to make themselves accessible to the researcher (Battaglia,

2008, Castillo, 2009). This could also result in the groups of respondents not being representative of the population of active participants.

On average, the surveys remained open for two months (this ranged from 1-3 months).

Survey administration was relatively straightforward with the exception of the Foldit survey. Approximately two weeks after I launched this survey, another researcher (an undergraduate from a UK university) posted a request for Foldit players to take part in some interview-based research exploring motivations for playing. Several Foldit players complained to one of the project scientists about the presence of too many external researchers and I was told that the link to my survey would no longer appear on the Foldit start-up page (which would have been viewed each time a player launched the game).

However, I was still free to post a link to my survey on the general forum. This illustrates some of the problems with online research and the fact that some project participants are not always comfortable being the subject of someone’s study. This can directly affect access to subjects, and it can also impact on the subsequent relationship between subjects and researchers. It also illustrates the importance of ‘gatekeepers’, and that their support may be required in order for the researcher to gain access to a potential pool of respondents.

Interviews of project participants

The next part of the data generation process involved semi-structured interviews with a sub-set of survey respondents, and with some of the scientists and developers26 involved in setting up and managing the projects. An interview gives the researcher the opportunity to explore issues in greater depth than a survey or questionnaire, and data is generated through the interaction between the interviewer and interviewee (Holstein and Gubrium, 2003). Indeed, Talmy (2011) describes an interview as a “collaborative achievement”. Interviews may also provide a way of verifying some of the findings of the surveys, and thus serve as tool for both methodological and data triangulation (Gillham,

2000). Interview questions should be ‘open’ giving the interviewee freedom to express their views, while the semi-structured nature of an interview can give the researcher flexibility and freedom to follow-up other topics of interest that come up during the discussion (Gillham, 2005). This was the approach taken in my research, and I aimed to design research questions that were general enough to enable respondents to talk around a point of interest, yet retained a focus on the overarching themes and objectives of this research. However, the researcher needs to direct the interview and can use prompts to steer the conversation back to the areas of research interest (Bryman, 2012).

The interview questions are listed in Appendices E and F.

Interviews have ‘traditionally’ been carried out face-to-face, or perhaps over the telephone (Shuy, 2003). However, when the focus of research is an online citizen project and participants are in numerous geographical locations and time zones, other approaches had to be considered. There are now a number of online and internet-based

26 By developers I am referring to those involved in designing, maintaining and updating the project software. Some of these individuals play a pivotal role in developing and managing online citizen science projects despite the fact that they may not be professional scientists or directly involved in utilising the data generated by citizen scientists. 76 research tools which can be used for conducting interviews (Jowett et al., 2011). These tools allow both synchronous interviewing and asynchronous interviewing (O'Connor et al., 2008).

Synchronous interviewing may be carried out using instant messaging tools, or increasingly, via Skype. Skype in particular is useful as the interviewee is visible, and so other more visual aspects of communication, such as facial expressions, can be seen

(Bertrand and Bourdeau, 2010, Hanna, 2012)27. Asynchronous interviews via email (also referred to as epistolary interviews) allow interviews to be conducted over time as well as over distance, and there is no need for transcription (Debenham, 2007). This approach allows a degree of flexibility as they may be easier to ‘schedule’ as the interviewee can respond to questions when it is convenient for them. Some researchers have also argued that giving interviewees this flexibility may mean that they take more time to consider their responses, as there is no pressure from an interviewer, or pressure to complete the interview in a specific time frame (Debenham, 2007, O'Connor et al., 2008). However, non-verbal cues are missing in online asynchronous interviewing, and the researcher is perhaps not as able to establish a ‘rapport’ as well as they could when interacting face-to- face with the interviewee (James and Busher, 2009b, O'Connor et al., 2008). All of the interview respondents were offered the option of being interviewed via Skype or via email. Most selected the latter (see Table 3.8). For two of the scientists involved in

Planet Hunters, I was able to carry out face-to-face interviews as they are based at a nearby university, and had previously invited me to visit.

One important question for a researcher who undertakes interviews is how many should be carried out? However, guidelines for determining adequate sample sizes for

27 A number of online tools are available for recording Skype conversations. I used a freeware MP3 Skype recorder http://voipcallrecording.com/MP3_Skype_Recorder . 77 interviews are not widely available in the literature (Baker and Edwards, 2012).

Undoubtedly, there are a number of external factors that will influence this such as subjects available, time constraints and what is expected in a particular discipline, but the researcher should also consider the purpose of their research, and what questions they are asking (Becker, 2012).

One approach that has been used to consider this issue is based on the concept of

‘theoretical saturation’. This can be been described as the process by which a researcher continues to sample relevant cases until no new insights are being gleaned from the data

(Bryman 2012). Some researchers have tried to quantify the appropriate number of interviews using theoretical saturation as a guide. For example, Guest et al., (2006) found that in their research based on 60 interviews, saturation was achieved after the first 12 interviews.

The number of interviews carried out as part of this research has been influenced by a consideration of theoretical saturation, and I aimed to carry out at least 10-12 interviews with citizen scientists from each project. How the subjects were recruited for interviews varied slightly between projects however, and was influenced by the number of survey respondents who were willing to be contacted again and who provided an email address.

In Foldit, all 20 respondents who expressed an interest in taking part in further research were contacted, and 10 agreed to be interviewed. In Planet Hunters, 63 survey respondents provided an email address, and approximately half of these were randomly selected and contacted. Eighteen participants agreed to be interviewed. Had the response to this call for interview subjects been poor, then the rest of the survey respondents who provided an email address would have been contacted.

In Folding@home I chose to email individuals who were members of two important sub- communities of participants who make up the active group of project participants. One of these communities in particular (computer hardware enthusiasts), has not been investigated in any detail previously (see Chapter Four, Section 4.2.3) and this study provided an opportunity to look at these participants in more detail. Thus, in the case of

Folding@home a structured sample of 60 participants was emailed. Structured sampling is used to explore the views of a particular group or sub-set of a population (Holliman,

2005). A total of 15 Folding@home participants agreed to be interviewed. Using this approach meant that only active participants across the three projects were included in the subsequent analysis.

During the thematic analysis of this interview feedback, it became clear that after 8-10 interviews there were few, if any, new codes generated. This was the case in all three projects.

Getting access to project scientists proved to be a lot more difficult than getting access to citizen scientists. In some cases my requests for interviews were not responded to, and in one project (Foldit) it took nearly 18 months before I was able to talk to key members of the project team. Many of the individuals concerned are busy running both research labs, as well as co-ordinating citizen science programmes, and it is often difficult to find the time to schedule interviews. All of these projects have small management teams ranging in number of from three to six, so the number of interviews carried out with scientists and developers is significantly smaller than the number of citizen science interviews.

Table 3.8 summarises the number of interviews for each project and how they were conducted.

Table 3.8 Details of the interviews carried out

Project Number of interviews Interview method Foldit 10 participants 9 email, 1 Skype 3 developers 1 email, 2 Skype Folding@home 15 participants All email 1 project scientist Both email 1 developer / moderator Planet Hunters 18 participants All email 4 project scientists 2 Skype, 2 in person

Skype interviews were recorded, although this was voice only and not video. However, immediately after the interviews, notes were made regarding body language, and other indicators of expression such as tone of voice, and level of interest and enthusiasm in the discussion. Interviews conducted by email varied in the amount of content and in the depth of responses. Some respondents gave very brief answers to the questions, while others gave very detailed and full responses, which occasionally highlighted other related topics. If an interesting issue was raised that I wanted to explore further, I would then go back to that participant and ask for more information, or ask further questions. A number of interviews of citizen scientists (six in total) involved several rounds of exchange and enabled me to gain a deeper insight into that individuals’ perspective.

3.4 Analytical framework

The analytical framework has been guided by both the case study approach and the mixed methods approach. In addition to the analysis of each separate data stream, the methodology requires that the results obtained be integrated in the subsequent analysis

(Bryman, 2006, Onwuegbuzie and Leech, 2005, Thomas, 2011a). This approach to data analysis has been referred to as ‘convergent concurrent design’ (Guest, 2013), and has been illustrated by Figure 3.4. During the analysis, evidence from multiple sources are

80 woven into a narrative account with the aim of presenting a ‘chain of evidence’ (Gillham,

2000, Thomas, 2011a).

Figure 3.4: Data Analysis (‘convergent concurrent design)

My experience as a participant observer in each project will be presented in Chapter Four, and provides the context for the feedback from the surveys and interviews that are presented in Chapters Five, Six and Seven. However, this research design permits comparison between cases, and this will be presented in Chapter Eight in addition to an analysis of the results for each case.

Each data stream was subjected to a different type of analysis. The methods chosen for these analyses and the rationale are outlined in the following three sections.

3.4.1 Online survey data

Much of the data from the four different data streams has been qualitative, although some quantitative data has been generated by the online surveys. The BOS tool automatically collates the quantitative data in tabular form.

A statistical analysis of some of this data was undertaken with the aim of exploring possible relationships between demographic variables (particularly educational background) and activity on online forums. Data relating to education, occupation, and number of hours spent participating on online forums and discussions were coded into discrete nominal categories and subjected to a Chi-square analysis using SPSS (Version

19). The Chi-square test measures the relationship between the observed and expected representations in each constructed category (Aron et al., 2011).

For each set of survey results, Chi-square contingency tables were executed that explored the relationship between the amount of time spent on online forums and discussions, with the level of general education, the level of education in STEM subjects (science, technology, engineering and mathematics), occupation, age, and length of time with the project. Gender was not considered as there were so few female survey respondents (46 out of 562), and it was felt that any resulting categories would have too few numbers to establish any statistical relationship. However, after conducting numerous tests for Foldit and Planet Hunters where the sample size was 37 and 118 respectively, poor representation in many categories was observed. As a general rule, categories in Chi- squared tests should occur at least five times (Ha and Ha, 2012).

Small sample sizes can be corrected for in Chi-square tests using the Yates correction or the Fisher Test, although they can only be applied to 2x2 chi-square tables (where the variables being considered only have 2 discreet categories each) (Richardson,1994). None 82 of my tests involved 2x2 tables, and attempts to reduce my overall numbers of categories to create 2x2 tables meant that most of these new categories were not as meaningful

(e.g. it was difficult to combine a range of different educational qualifications into two categories). A reliable statistical analysis was therefore not possible with the Foldit and

Planet Hunters data. As Folding@home had a larger sample size (407), Chi-square tests yielded better representation in the sub-categories. The results of the tests carried out for Folding@home will be presented in Chapter Six (Section 6.2.7).

The qualitative feedback from the online surveys has been subjected to a content analysis, and each open-ended question has provided the ‘unit of analysis’ (Bos and

Tarnai, 1999). In content analysis, textual data is explored inductively for emerging themes or ‘meaning units’ that relate to the same central meaning (Graneheim and

Lundman, 2004, Elo and Kyngäs 2008). These themes can be grouped into content or coding units which can then be counted and subsequently expressed and analysed quantitatively. Coding units should be discrete (with no conceptual overlap) and be neither too general nor too specific (Bryman, 2012).

Coding themes were devised for each open-ended question after reading and re-reading the responses and inductively identifying keywords, topics and sentiments that were expressed. An initial list of codes was then examined to see if there were any obvious groupings and a final, more concise coding frame was constructed. The occurrence of each code (or theme) was counted and this has formed the basis of the graphical representation of the open-ended survey data in each of the project chapters. This approach was chosen as it enables qualitative data to be illustrated in such a way, and permits a more immediate and concise appreciation of this data without the need for reading the entirety of the survey responses. It also illustrates the importance or

83 predominance of certain themes which will help to address the research questions, and facilitate inter-project comparison. The codes are outlined in the project chapters.

3.4.2 Interview data

The interview data was subjected to thematic analysis. This is a widely used qualitative analytic method for inductively identifying, analysing, and reporting patterns (or themes) within data (Bryman, 2012, Silverman, 2006). Gillham (2005) defines a ‘theme’ as a horizontal category, something that exists as a kind of ‘sub-plot’ within the main narrative. The identification of themes is thus a form of pattern recognition that allows the researcher to go further, ultimately using the emerging themes as categories for analysis (Guest et al., 2012).

Despite its widespread use, there are relatively few ‘guides’ to this methodology available in the literature (Braun and Clarke, 2006, Ryan and Bernard, 2003). Braun and Clarke have written a comprehensive and accessible guide and I have followed their approach which can be broken down into six separate stages (Braun and Clarke, 2006).

1. Familiarising oneself with the data (transcribing, noting, reading and re-reading).

2. Generating initial codes (coding interesting features of the data in a systematic

fashion).

3. Searching for themes (collating codes into potential themes).

4. Reviewing themes (checking if the themes work in relation to the coded extracts).

5. Defining and naming themes (refine the specifics of each theme, generating clear

definitions and names for each theme).

6. Producing the report (final analysis involving the selection of extracts and relating

these back to the research questions).

Participant responses from each of the three projects were analysed separately, and responses from citizen scientists were analysed separately from those of the project scientists and developers. Therefore, six separate thematic analyses were carried out in total.

For each analysis, all of the interview responses were treated as one body of data, rather than subdividing it according to the responses to each individual question. One of the most commonly made errors in thematic analysis is to use the interview questions as default codes (Braun and Clarke, 2006), and I wanted to avoid this. Furthermore, the responses to individual questions are not always related to the actual question that was asked, and many respondents go off-topic and highlight other related (or unrelated) issues.

The identification of coding themes in both thematic and content analysis is subjective, and can be influenced by researcher bias (Bos and Tarnai, 1999). In order to address this issue, some researchers employ investigator triangulation and seek the agreement of co- researchers of the coding frames in order to improve the reliability of the analysis (Carr et al., 2009). This is known as ‘inter-coder reliability’ and it can be expressed as the percentage agreement between the coding results of two different researchers coding the same body of data (Bryman, 2012). A common ‘rule of thumb’ requires that researchers should agree at least 80% of the time (Bayerl and Paul, 2011), although others put this figure at 70% (Guest et al., 2006). In order to explore inter-coder variability in this interview data, a small sample of Foldit interviews was given to a colleague along with a detailed description of the coding frames. An agreement of approximately 70% was achieved for this selection of material.

The use of measures of inter-coder reliability can be problematic however, and values of

80% or more can be hard to achieve (Bayerl and Paul, 2011). In a meta-analysis of 96 studies that reported inter-coder reliability, Bayerl and Paul (2011) found that this value varied from 13-100%, and that it was influenced by the number of codes included (more codes meant less reliability) and whether adequate training was provided to coders.

Bryman (2012) states that another important consideration in establishing research reliability for lone researchers is ‘intra-coder reliability’. This can be expressed in the same way as inter-coder reliability, only the comparison is made between two examples of coding of the same material by the same individual at different points in time. In order to explore this, I re-coded a small selection of Foldit interviews six weeks after they were coded for the first time. An agreement of over 90% was achieved for this selection of material.

3.4.3 Participant-observer notes

My experience as a participant observer has been recorded in the form of field notes and annotated screen shots (see Table 3.6). Like the qualitative survey responses and interview feedback, this material has been reviewed and examined for emerging themes in an inductive manner and a narrative has been constructed that seeks to address the research questions. A preliminary analysis of the participant-observer material was conducted shortly after joining these projects and these observations have informed the content of the online surveys (see Figure 3.3). This preliminary analysis also forms the basis of the description of each project that is presented in Chapter Four.

After the completion of each survey, another consideration of the participant-observer material was undertaken and this, along with some of the preliminary findings from the survey, was used to inform the follow-up interview schedule. This second stage of 86 participant-observation analysis has informed a more detailed consideration of my experience as a participant, which is presented at the end of each project section in

Chapter Four. This has been used to create a context for the survey and interview feedback, and to make an initial attempt to address some of the research questions.

In a final analysis, participant observations over the total study period have been considered along with the other data streams (see Figure 3.4) and presented in Chapter

Eight. All of the data collected has been integrated and used to address the research questions.

3.5 Conclusion

While this chapter has dealt with the methodological approach and methods used, some consideration must be given to the overall ‘trustworthiness’ and rigour of this research

(Lincoln and Guba, 1986). Shenton (2004), states that four criteria need to be addressed in order to demonstrate trustworthiness in qualitative research projects. The first of these is credibility and this requires that the researcher demonstrates that a true picture of the phenomena being studied is presented. This can be addressed by selecting research methods that are well established, the use of research triangulation, and by developing some familiarity with the subject under investigation before data collection takes place (Silverman, 2010).

The second is transferability, although with the caveat that this may be difficult to demonstrate because the findings of qualitative studies are specific to a small number of environments. This can be alleviated however, by providing sufficient and detailed contextual information about the research (Thomas, 2011a). The third is dependability which relates to the repeatability of the work. While this too is problematic due to the highly specific nature of much qualitative research, it can be addressed by providing a 87 detailed description of how the research was executed (Shenton, 2006). The final criterion is confirmability, in which the researcher takes steps to demonstrate that that their findings have emerged from the data, and not from their own predispositions. Data triangulation can help to promote confirmability, as can a detailed outline of the research steps and decisions taken in the form of an ‘audit trail’ for other researchers to examine

(Jensen and Holliman, 2009). The researcher must also been keenly aware of the potential for researcher bias (Gillham, 2000).

Attempts have been made to fulfil the above criteria in this study, and this has been facilitated by using a multi-method, case study approach. Established data collection methods have been employed, and a substantial effort has been made to gain an in- depth understanding of the selected cases through the first phase of participant observation, before the rest of the data was collected. These steps may help to establish the credibility of this work. Multiple strands of evidence have created opportunities for data triangulation, which may help to establish credibility as well as confirmability. An attempt has been made in this chapter to give a detailed account of the research methods that were used, how they were used, and why they were used, providing an audit trail that can be evaluated by other researchers. This detailed account of methods and methodology has thus aimed to establish the dependability and transferability of this research.

Chapter 4: Description of the projects

This chapter provides an overview of the three projects selected for this research. For each project, the following information will be presented:

 when and where the project was developed;

 a description of the project task(s);

 an estimate of the number of active participants;

 opportunities for interaction between participants (including interaction between

citizen scientist volunteers, and between citizen scientists and members of the

project team).

The above information has been collected through my initial observations and early participation in each of the three projects (see Chapter Three, Section 3.4.3). Sustained participation in these projects has enabled a more detailed exploration of aspects of participation, and each project section will conclude with an overview of my experience as a participant, and how these observations can begin to address the research questions.

4.1 Foldit (www.fold.it)

Foldit is a project in which a challenging scientific problem, the creation of accurate protein structure models, has been turned into a multiplayer online citizen science game.

The rationale for the development of Foldit was to harness the collective problem-solving abilities of non-experts to accelerate progress in understanding the three-dimensional structures of protein (Cooper, 2011). Foldit was developed at the University of

Washington in Seattle by a group led by Professor David Baker 28at the Department of

Biochemistry in collaboration with researchers at the Centre for Game Science led by

28 The work of the Baker Lab focuses on the prediction and design of protein structures, and interactions between protein molecules http://www.bakerlab.org/index/. 89

Professor Zoran Popovich. It was released to the public in May 2008. Project scientists and developers work closely to ensure that Foldit puzzles address the research needs of the Baker Group, while still appealing to Foldit players (Cooper, 2011). Foldit has had some success recently which has resulted in a number of significant publications in high profile science journals where Foldit teams have been listed among the co-authors (Eiben et al., 2012, Khatib et al., 2011a, 2011b) (see Figure 4.1). Consequentially, a number of news and magazine articles have been published praising Foldit for its approach to

‘crowdsourcing science’ and for opening up the scientific research process to members of the public (Hand, 2010, Sansom, 2011, Bohannon, 2009, Timmer, 2011, McGonigal, 2011).

Figure 4.1: Foldit publication in Nature Structural and Molecular Biology (Khatib et al.,

2011b) featuring two Foldit teams (underlined) as co-authors.

4.1.1 Underlying science and project task

Proteins are central to biochemistry and are the primary chemical for almost all cellular processes. Much can be learned about a protein’s function once its structure is understood, and this can be applied to the study of common diseases, or to the development of innovative biological molecules (Cooper et al., 2010, Dill and MacCallum,

2012). Foldit players enlist human three-dimensional problem solving skills and online manipulation tools based on computer algorithms, to produce accurate models of protein structures that have previously been unknown (Khatib et al., 2011b). Players can play individually, or within a team, and compete against one another within a points system.

Protein structures that are confirmed to be the closest to their ‘natural’ configuration

(that is one that requires the least amount of energy) are awarded a greater number of points. Despite the competitive aspect of the game, players also work together cooperatively and / or collaboratively (see Section 2.4 for a definition of these terms) to solve the protein puzzles.

Foldit is a complex game and can be difficult to learn compared to other multiple–player online games (Andersen et al., 2012). Before a player can compete and work on protein puzzles where the structure is unknown (known in Foldit as ‘science puzzles’), there are a series of tutorial or ‘intro’ puzzles (32 in total) that guide the player through the various game tools available. However, there is no requirement that the individual complete all of these puzzles. Intro puzzles are based on proteins where the structure is already known. Figure 4.2 illustrates one of the tutorial puzzles.

Figure 4.2: Third Foldit ‘intro’ (tutorial) puzzle. In this puzzle, the ‘shake’ tool is being introduced.

2. Message bubble 3. Global chat window

1.Protein molecule

The protein molecule (1) can be seen in the centre of the screen, and this can be moved around and manipulated with the game tools. The message bubble (2) explains the tool being introduced. The player can interact with others on the synchronous global chat window (3). Once confident with the structure and tools of the game, a new player is then able to play the science puzzles (Figure 4.3).

The protein molecule (1) is now more complicated, and there are a greater array of game tools available (2). Box 3 displays the title of the puzzle, current player rank, while Box 4 illustrates the team ranking for this particular puzzle. In addition to the global chat window, the team chat window is also shown (5).

Figure 4.3: Foldit Science Puzzle (solo). 3. Puzzle name, current score and rank

6. Recipes 4. Team rank in this puzzle. library

5. Global and team chat windows are open

1.Protein molecule. 2. game tools.

During the last few years Foldit players have developed a way of automating moves in the science puzzles. This has been achieved by coding sequences of moves using the scripting language Lua. These sequences of moves have become known as ‘recipes’ and once developed, can be shared with other Foldit players (Cooper et al., 2011). Some of these recipes have been adapted and improved upon a number of times, and Foldit players often build up their own libraries of their preferred recipes (see Figure 4.3, Box 6). Those who create recipes can receive recognition for this work. The use of recipes has significantly changed the way many participants play Foldit, although some choose not to use them and remain committed to ‘hand-folding’.

Foldit players can either work on a puzzle until it is completed to the best of their ability, these are known as ‘soloist’ puzzles, and players achieve a ‘solo rank’ based on their performance in these puzzles. Alternatively (or additionally), they can work on other players’ incomplete puzzles until a good score is attained. These are known as ‘evolver’ puzzles, and a player can achieve an ‘evolver rank’ based on their performance in these puzzles.

4.1.2 Foldit players and teams

It is relatively easy to sign up to play Foldit and to set up a user profile. Once an individual has registered to play, a player page is set up which records all of their activity on the game (Figure 4.4 shows my player profile). For example, when they last played (1), which tutorial puzzles they have completed (2), which science puzzles they have completed (3), points awarded (4) and other ‘achievements’ such as how many moves they have carried out, or whether they have won any challenges (5). The player can also edit their profile to include more personal information about where they live, what they do, other hobbies and interests (6).

Figure 4.4: Player profile page

4 1

They may also post a photo or avatar, and directly send emails to other Foldit players.

Player profiles can be accessed through the Foldit website and are within the public domain.29

As a player works their way through the tutorial puzzles, these achievements become

‘unlocked’ on the players profile page, and others can view what stage a player has reached. Other achievements can be ‘unlocked’ and these are related to performance in certain types of puzzles, the creation of recipes, or whether a suggestion has been taken on board by the project team. Opportunities to ‘unlock’ such achievements have been shown to be a powerful motivator to participate in other types of computer games (Yee,

2007, Williams et al., 2008).

Teams also have profile pages (Figure 4.5) that list the members of the team and the team achievements. While there are hundreds of registered teams, most are inactive or have very little registered activity. Foldit is currently dominated by approximately 10 active teams and some of these have been around for several years, e.g. the ‘Void

Crushers’ , ‘The Contenders’, ‘Anthropic Dreams’ and ‘Go Science’. A new player can approach a prospective team that they wish to join, but membership is not automatically given, and players may have to demonstrate their skill or commitment to the game.

Some teams approach promising new players and invite them to join. However, any player can set up their own team. Each profile page contains some background information about the team and who the manager and main point of contact is (1).

Overall rank and score are listed (2), as are ranks for each type of puzzle (3). Recent activity is also tracked (4).

29 Directory of all registered Foldit players: http://fold.it/portal/players. 96

Figure 4.5 Example of a team profile

Both player profiles and team profiles have been an invaluable source of information about the active playing community in Foldit.

4.1.3 Active playing population

While there are over half a million registered participants, it appears that many of these individuals have not actually played Foldit, or completed any of the tutorial puzzles.

There are no recorded points, lists of completed tutorial puzzles, or playing activity on most of the profile pages that were examined. A closer analysis of the leader boards, player profile pages, online forum postings, and in-game player statistics (the number of individuals playing each puzzle are displayed), suggests that there is a relatively small population of active players. From my observations, I estimate that the active Foldit playing community is in the region of 200-300. This active group of participants includes players who are playing at least 2-3 times a week and interacting regularly on global chat while they are playing. The majority of active players also belong to the top-ranking

Foldit teams. Active players have been identified by their usernames and their playing activity has been tracked on individual and team profile pages.

From this active playing community, a small group of dedicated ‘core’ players has emerged. These are the players who have been playing for at least 2-3 years and play daily. They frequently participate in online scientist’s and developer’s chats, get involved in reporting and fixing problems with the game, create content for the Foldit wiki, moderate the online global chat, manage teams, help to mentor and develop new players, and write many of the recipes. Some members of the core group have contributed to some of the key external scientific publications. These players are easily recognised by their usernames, and their sustained appearance and contribution to Foldit is documented on the website (all forum discussion threads from 2008 onwards can be viewed). I estimate that the core group numbers approximately 20. Several members of the Foldit development team have corroborated my estimates of both the active playing

98 population and of the core group via interviews, although actual numbers are difficult to verify.

4.1.4 Interaction between Foldit participants

Interaction between the participants of Foldit (including interaction between players and between players and members of the Foldit project team) can take place on the Foldit project website or via the internet relay channel or ‘chat,’ during the online game sessions. One of the main areas where Foldit players can interact with each other and with members of the Foldit project team is on the website forum (Figure 4.6). Many of the discussion threads relate to how to play the game, an introduction to protein folding, information about competitions, new versions of the game software, official policies relating to intellectual property and patents, technical issues as well as general discussions instigated by players about the game (e.g. how can we help new players? How can we improve the game?). The majority of discussions on the Foldit forum is related to the game and seldom stray into other topics. Many discussion threads are highly technical in nature, and there appear to be some Foldit players who have learned about protein folding and regularly use the appropriate scientific language (Figure 4.6).

The website contains a blog that is written by the project scientists and developers, and players can comment on these posts which gives them the opportunity to communicate directly with scientists and developers. Players can also communicate directly and in real- time with the project team via ‘scientists’ chat’s or ‘developers’ chats’ which are held approximately every two or three months. Any player can participate. Discussions were usually of a technical nature relating to updates to the game, or the development of new puzzles. The transcripts of these chats are posted on the website30. Players can also

30 Archive of online chats between players and members of the project team http://fold.it/portal/chats. 99 report problems and make technical suggestions to the project team in the ‘Feedback’ area of the website31.

Figure 4.6 Example of Foldit forum discussion

31 ‘Feedback’ area of the Foldit website: http://fold.it/portal/feedback. 100

Much of the interaction between Foldit players occurs in real-time while playing via the internet relay chat. During the game, a player has the option to access three different chat channels:

1. Global chat – where any player can talk to any other player who is currently

playing online. These discussions can be viewed by anyone playing.

2. Veterans chat – for more experienced players (those with 150 points or more).

This currently (2014) includes approximately 120 players.

3. Team chat – where members of the same team can interact. Each team has its

own discussion space and it cannot be viewed by players who are not in that

team.

Foldit has a set of community rules32 that players are expected to adhere to. Failure to behave in an appropriate manner can result in the players being suspended from the internet relay chat temporarily, or in more serious circumstances, their account may be deleted. These rules state that racism, sexism, making threats, posting spam, releasing personal information about others, discussion of illegal activity such as online pirating, general rudeness etc. will not be tolerated. They also remind players that participants come from a variety of nationalities and cultures, and that there may be young people taking part. There are occasionally instances of individuals ‘trolling’ or being disruptive on global chat although it is not always clear whether these are regular players. In such situations, forum moderators can step in and remove (or ‘kick’) a player from chat.

32Foldit community rules and online code of conduct: http://fold.it/portal/communityrules. 101

4.1.5 Participant experience

Learning to play

While not compulsory, players are strongly advised by the project team to complete the

32 tutorial puzzles before playing the science puzzles. As soon as I had registered to play I began the tutorial puzzles. The first few were straightforward and were completed with relative ease within 5-10 minutes. However after about ten tutorial puzzles, the level of difficulty appeared to increase sharply. Some of the puzzles took a number of attempts to complete as the instructions were not always detailed enough, and it was difficult to understand fully what the new task entailed. The amount of time spent on each tutorial puzzle became progressively longer. I spent up to an hour on some of the later puzzles.

While playing, I watched the global chat window and observed how other more experienced Foldit players emphasised the importance of completing the tutorials to other new players who would use the global chat to ask for help. New players were also referred to the wiki where there was more information on the tutorial puzzles, and short animations demonstrated how to complete each puzzle successfully33. I regularly consulted these during my own learning experience, and these helped me to complete some of the more difficult puzzles.

Joining a team

Once my online questionnaire had been launched in late June 2012, I was approached by another player to see if I would like to join the team ‘Go Science’. He suggested that I may gain more of an understanding of the game and may benefit from the help of others while learning to play. Once a member of the team, I was able to take part in the team

33 A short video showing how to complete tutorial puzzle 4.3 ‘Flippin’ sheets’. These guides are available for all of the tutorial puzzles. http://www.youtube.com/watch?v=RDLNivggADE 102 chat during play which helped to provide an insight into the interaction between Foldit team members. Advice about the tutorial puzzles and playing strategies was also shared.

Some of my team-mates demonstrated a very high level of commitment to Foldit. I would sometimes play in the morning and find that one or two of my team-mates who were based in the US still playing into the night. If I was playing in the afternoon then some of my US team-mates would be playing first thing in the morning before they went to work.

Watching the interaction within my team and the level of technical understanding exhibited by some of the individuals, inspired me to persevere with the tutorial puzzles so

I too could reach their level of competence and be able to take part in more specialised discussions. This ‘aspirational’ motivation to be as competent as my team mates was strong, and motivated my persistence with the tutorial puzzles. However, learning to play could be frustrating at times even with help from others, and I began to understand why many prospective players abandon the game in the early stages. After weeks of learning how to play it became evident that the Foldit tutorials represent a significant hurdle that must be overcome if an individual wants to make a contribution to the project, and they present a high barrier to participation. This has an impact on motivation to play Foldit, particularly motivation to sustain participation, and those with the patience (and time) may be more likely to persevere with the learning process. These observations have shed some light on the motivations that can sustain participation (RQ2).

Playing Foldit science puzzles

Playing the science puzzles proved to be more enjoyable than playing the tutorial puzzles and this was partly due to the competitive aspect. I found this aspect of the game motivating, and enjoyed seeing my points and rank increasing although I was not getting

103 anywhere near the top 100 initially. Some of the puzzles are only open to new players

(those with less than 15 points) and the structure for these proteins is already known. I have managed to rank in the top ten for two of these puzzles. I did not complete anyone else’s puzzles (evolver puzzles) as I felt my level of skill was insufficient.

I used a number of recipes while playing Foldit recommended by my team-mates, and seldom used ‘hand-folding’ (see Section 4.1.1). However, I did not write any recipes as this would have required learning how to programme in Lua. My reliance on recipes meant that I gradually forgot about the some of the basic tools that were outlined in the tutorial puzzles, and I was guided purely by points. I felt that I did not have an understanding of what the recipe was actually doing in terms of modifying the structure of the protein molecule. Using recipes disconnected me from the underlying science to some degree. Whether an understanding of the science is important for other Foldit players will be considered in greater detail in Chapter Five (Section 5.2.1). While the tools of Foldit are based on underlying scientific principles and use the appropriate terminology, other citizen science games have made an effort to ‘distance’ the task from the science. For example, Phylo34 is a Tetris-style game that seeks to understand the phylogenetic relationships between different species. The underlying science has been deliberately removed from the highly stylised puzzle-based task (Kawrykow et al., 2012).

Observations of interactions between participants

My observations have not only enabled an insight regarding how and where players communicate, but also the purpose of these interactions, thus addressing RQ 4 (how and why do participants interact online). While observations were initially made on the project forum, most of the interaction between players was observed on the global chat.

34 The game Phylo was developed in 2010 by scientists at McGill University in Montreal http://phylo.cs.mcgill.ca/. 104

During some of this interaction, players used technical terms relating to the biochemistry of protein folding, or would describe some of the underlying biological process that the recipes were altering, thus demonstrating some detailed knowledge of protein biochemistry. Players helped each other with the puzzles, and in some cases, worked collaboratively in solving a particular puzzle. In addition to these technical discussions, players use the global chat get to know one another by discussing more personal topics.

Most of the interaction I had was with other team mates via the team chat window during online play. Two individuals in particular were extremely helpful and I learned a great deal about the mechanics of the game from them, and how some players take an active role in coaching new players. My team-mates worked together regularly on the same puzzles, posting their results and arriving at a solution collaboratively. While neither had a background in biology (one was an economist and the other worked in IT), they were confident in their use of the language of protein biochemistry and in the application of what they learned in an area outside of their expertise. In addition to discussing Foldit, we also had conversations about other topics such as our families, jobs, and hobbies. I found team chat to be a very friendly and pleasant place to interact, and this was perhaps one of the most enjoyable aspects of playing and one which kept me returning to Foldit thus sustaining my participation in the game.

I also interacted with other Foldit players via the forum. In addition to my first forum postings relating to my survey and research interest, I also took part in other discussion threads including one relating to the tutorial puzzles where one of my team-mates suggested that they may need to be simplified. I also posted links to papers or articles where Foldit has been mentioned, and to other online citizen science projects that I

105 thought would be of interest to other players. However, I had little response to these posts.

In addition to real-time chat and the forum, I observed the ‘Feedback’ area where problems are reported and suggestions made to the project team. The number of individuals interacting here is much smaller than on the global chat or the team chat, and

I observed many of the same players (numbering approximately 10-15 individuals) over time. These individuals were usually from the core group of players. This small group has a direct connection to the project team, and are present regularly at scientist’s and developer’s chats. From the transcripts of these chats (see Figure 4.7) and an examination of player profiles, it is clear that some of these individuals have been with the game for several years and have been instrumental in guiding some aspects of the game’s evolution and development.

In this scientist’s chat, beta-helix and @CEiben are members of the scientific team and the other four participants are players. Only small numbers of players (4-10 on average) generally participate in these chats, although there may be more players who observe and do not contribute.

While the project team are responsible for the overall maintenance of Foldit, setting the puzzles, analysing the results, and writing them up in the scientific literature, there appears to be little interaction between the project team and the majority of Foldit playing community. Many of my observations suggest that the community of players demonstrate a degree self-sufficiency and independence from the scientists and developers. The project tasks are carried out by the players, and the interaction

(including co-operative and collaborative efforts) appears to occur more between players, than between players and project team. 106

Figure 4.7 Transcript of Scientist chat.

Roles within Foldit

While not everyone can, or wants to be, in the core group, I observed a variety of different roles that can be fulfilled by participants in the group of active players. After observing the forum discussions, user profiles, internet relay chat, and through playing, I identified eight different roles that active players were fulfilling (Table 4.1), and some

107 players may undertake more than one of these. Thus, active contribution to the project can be further characterised addressing RQ 5 (how can contribution to online citizen science be characterised?).

Table 4.1: Player roles within Foldit

Players who work through a puzzle from start to finish, although this name is Soloists slightly misleading as players often work together with fellow team members, or members of the wider Foldit community to complete a puzzle. Most of the active players work on solo puzzles at some point. These players help to complete other player’s puzzles. They are often the most Evolvers proficient players and usually have very high puzzle scores. Players can be identified in a separate leader board for evolvers. Approximately 50 active players work on evolver puzzles. Players who organise their team, and may also recruit new members, and Team ‘managers’ moderate discussions on team chat. Actual numbers are difficult to ascertain as team discussions are closed those outside the team. Given the number of teams and players, this figure is estimated to be 10-20 in total. These players moderate discussions in the global chat area. They have the Global moderators power to remove players from the chat if they violate the community rules. There are 4-5 moderators, and they appear to change infrequently. Specialised players who write ‘recipes’, scripts written in the programming Scripters language Lua, which may be used to automate the players moves. A scripter is awarded points based on the ratings other players give to the recipes, and a player can unlock achievements this way. Proficient recipe writers can become well-known among the Foldit community, and there are approximately 15-20. These are usually players who have been with a project for a relatively long Hand Folders period of time, and began folding before any of the tools for automation were introduced. They do not rely on the use of recipes and prefer to use the manual tools available. Successful hand folders are often revered by some of the other players. There are very few of these players, and it is estimated that there are approximately 5-10. Many players help new players to learn about tools and strategies, but some Instructors / ‘teachers’ have gone to greater lengths by putting detailed advice about Foldit tools and strategies, as well as simulations of the tutorial puzzles on the wiki. Numbers are hard to ascertain as many players help others. However, approximately 5- 10 players have been responsible for putting much of the information on the player wiki. Some of the more technically proficient players monitor and report software Technical experts bugs and other technical issues, and may help the community fix these technical glitches when they occur. Approximately 10-15 players take part in these technical discussions.

108

4.1.6 Foldit: summary

Foldit was one of the first online citizen games developed, and it has demonstrated that this format of online citizen science project is capable of generating significant research results (Khatib et al., 2011a, 2011b). However, there are high barriers to participation and through my analysis I found aspects of the game difficult to learn. Despite the large number of registered players, Foldit consists of a relatively small playing community of dedicated individuals, with a ‘core’ group of highly active participants who have been instrumental in the way Foldit has developed over time (Cooper, 2011). Interaction with other players is key to learning how to play, and is also important for the completion of the science puzzles. In addition to completing the puzzles, other project tasks have been identified. These observations have helped to address how and why participants interact online (RQ4) and the different ways that participants can contribute to a project through a variety of project roles (RQ5). Factors affecting motivation such as the difficulty of learning how to play, and the competitive nature of the game (RQ 2) have also been highlighted.

4.2 Folding@home (http://folding.stanford.edu)

Folding@home was one of the first distributed computing projects in the biological sciences, and is based at the Chemistry Department at Stanford University in California

(Sansom, 2011). It was developed in October 2000 by Professor Vijay Pande, and his group continues to manage the project and process most of the results (Beberg et al.,

2009, Sansom, 2011). Folding@home can run on some game systems and graphics processing units in addition to PCs, and until November 2012, it could also run on the

Sony Playstation 3. The use of games and graphics processing units greatly increases the processing capacity of the project (e.g. a PlayStation 3 processor was 20 times faster than

109 a standard PC) and it is this capability that has made Folding@home one of the fastest computing systems in the world (Lane et al., 2012, Pande, 2003).

4.2.1 Underlying science and project task

While Foldit seeks to ascertain the final structure of a protein, Folding@home seeks to understand the whole process of folding. Before proteins can carry out their biochemical functions, they assemble themselves or ‘fold’ into a final three-dimensional structure that is specific to each individual protein (Dobson, 2003). This involves their transformation from a ‘string’ of amino acid molecules, to a complex structure of helices and sheets held together by different types of chemical bonds (see Figure 4.8). The process of protein folding is not fully understood despite being fundamental to nearly all biological functions

(Dill and MacCallum, 2012).

Figure 4.8: Process of protein folding. A simple chain of amino acids on the left is transformed (folded) into a complex three-dimensional protein molecule (Hunter, 2003)

110

If a protein ‘misfolds’ there can be serious consequences for an organism. For example, in humans, Alzheimer’s disease, Creutzfeldt Jakob Disease, Huntington’s disease, motor neurone disease and some cancers all occur as a direct result of protein misfolding

(Beberg et al., 2009, Hunter, 2003).

In the cell, protein folding occurs incredibly quickly – within milliseconds or microseconds.

Folding@home enables this process to be slowed down considerably so that folding is simulated at a rate thousands to millions of times slower than it occurs naturally (Lane et al., 2012). This allows scientists to examine protein folding more closely, and to study aspects of folding that cannot be easily studied in laboratory experiments. However, advances in simulation have led to new experimental approaches (Pande, 2003) and the combination of simulation and experiment has greatly advanced knowledge of the process of protein folding over the past 10 years (Bowman et al., 2011).

Currently, Folding@home is investigating protein misfolding in relation to Alzheimer’s disease (specifically in the formation of protein aggregates in the brain), Huntington’s disease, and cancer (specifically misfolding in a molecule known as P53, a tumour suppressor gene). Another key area of research has been using computational methods to design new drugs, especially for Alzheimer’s. The Pande Group, as well as other research groups, has also used Folding@home to investigate diabetes, Parkinson’s disease, brittle bone disease, and some of the underlying mechanisms associated with the development of viral diseases. Since its launch, there have been over one hundred scientific publications based on the results of Folding@home35.

35 List of published papers based upon the work output from Folding@home. http://folding.stanford.edu/home/papers 111

If a participant wishes to find out more about the science relating to protein folding, there are a number of resources on the Folding@home website including background information about the science, FAQs about the project, interviews with the scientists, links to all of the resulting publications (although not necessarily full access), links to external resources, and a blog written by Vijay Pande36.

4.2.2 Participating in Folding@home

Downloading the Folding@home software is relatively straightforward, and once installed, participants can view progress of the current work unit, and learn more about the protein being explored in a web browser (Figure 4.9). There are also links to the latest news about the project, and background material relating to the underlying science. The participant can also control how much processing power they would like to donate to the project (e.g. light folding power is recommended for lap-tops).

Figure 4.9 Folding@home progress page

36 The science behind Folding@home: https://folding.stanford.edu/home/the-science/. 112

Once an individual (or team) registers then a profile page is set automatically set up, and these can be viewed by anyone. These are less detailed than Foldit profiles but display the number of points (1), date the last work unit was completed (2), rank (3), team affiliation (4) and the number of clients a participant runs (5). Figure 4.10 shows my profile. I am not affiliated with any team, so my points go to a ‘default team’. My profile page also shows that I have a very low ranking, although this rank is given out of the total number of participants who have ever run Folding@home since its launch in 2000. The number of current participants is much lower and is discussed in Section 4.2.4.

Figure 4.10 my Folding@home participant profile

2 1

5 3

113

As in most distributed computing projects individual participants are awarded points for each processing unit of power that they donate to Folding@home, and individuals or teams can compete for the most points. An element of competition is one way that distributed computing projects can make participation more enjoyable and interactive

(Choi, 2005). Folding@home teams often try to recruit members through the

Folding@home project forum, and the Folding@home website has a leader board where the top 100 individual participants and top 100 teams are listed. Participants are free to join any team that is registered with the project. Statistics relating to accomplishments are important to some participants (Bohannon, 2005, Larson et al., 2009) and there are a number of websites that maintain up-to-date figures for the various distributed computing projects including Folding@home37.

A small number of Folding@home participants (approximately 3-4 individuals) have taken a more active role in the project by helping to moderate the forum discussions. Several participants have contributed to the project by helping to translate pages of the website into other languages, and helping to keep the FAQs up to date. Participants who have suitable computing experience can help make improvements to the software. These volunteers are referred to as the Beta Testers and the support they contribute is in addition to the professional software developers who have been employed by the project. There are approximately 30 individuals in the Beta Testers (as of 2014).

37 There are a number of websites that enable distributed computing enthusiasts to keep track of projects they are participating in and who the top contributors are, for example: http://www.teamocuk.co.uk/index.php?s=8f315d852c601368eb111539388a9393 ; http://stats.free- dc.org/stats.php?page=index.

114

4.2.3 Hardware enthusiasts and ‘overclockers’

As success on the leader board requires that a very high numbers of computer or graphics processing units are donated, distributed computing attracts a particular group of participants known as ‘overclockers’ (Bohannon, 2005). They are computer hardware enthusiasts who build custom machines with the aim of maximising their processing power (Colwell, 2004). Occasionally, individuals compete to see whose machine is able to produce the most processing power, and one way to measure performance is through participation in distributed computing projects. These projects thus provide a

‘benchmarking’ tool to the overclocking community. Little has been written about overclockers and their contribution to distributed computing projects, but one estimate states that they may contribute over half of the processing power to all distributed computing projects (Bohannon, 2005).

There are numerous overclocking and hardware enthusiast communities who interact online38. Distributed computing projects are discussed on overclocker and enthusiasts’ forums, and these forums may serve as an important vehicle for increasing publicity about a particular project and for recruiting new participants. Occasionally, teams of overclockers come together for specific competitions, for example, the ‘Chimp Challenge’ is an annual competition between teams on Folding@home to see who can process the most work units over a given time period39.

Some overclockers invest significant amounts of money in their machines (sometimes many thousands of US dollars) and running them may require a large amount of electricity. Many ‘rigs’ incorporate multiple processing components and require

38 Some examples of overclocking forums and communities can be found here: http://forums.overclockers.co.uk/, http://www.overclock.net/, http://www.overclockersclub.com/. 39 Chimp Challenge announcement at one of the overclockers forums: http://www.overclock.net/t/1377824/official-chimp-challenge-2013 (retrieved 29/04/14). 115 sophisticated cooling systems (Figure 4.11). These participants may therefore make a significant financial investment (albeit indirectly) to a distributed computing project.

Figure 4.11: Customised ‘folding rig’. The use of neon lighting and purpose-built display cases are there to add to the aesthetic appeal of the machine (image from the EVGA Folding Forum40).

It is likely that overclockers make a significant contribution to Folding@home although it is difficult to ascertain with any accuracy what percentage of project work units they complete. Many of the largest teams are affiliated with manufacturers of computer hardware (e.g. EVGA, a manufacturer of hardware and graphics processors) so it is highly probable that members of those teams have an interest in overclocking. Other teams have a reference to overclocking in their name, for example, ‘Overclockers Australia’ and

‘Overclockers Club’.

40 A discussion relating to differing hardware set-ups and a gallery of images can be viewed at the EVGA overclocking forum. http://forums.evga.com/tm.aspx?m=28921 116

4.2.4 Participant numbers

There are currently (August 2014) over 175 000 computers (clients) running

Folding@home although this is dynamic. While the number of computer clients can be accurately ascertained, it is difficult to know with any certainty how many individuals this translates to as some participants run the software on several machines or processing units. For example, one of the top ranking contributors has 34 active clients according to their profile page41. Professor Pande estimates that there are approximately 100 000 individuals participating in Folding@home (Pande, 2013). However another source which is based on user statistics obtained from the Folding@home website every three hours,

Kakaostats42, gives a much lower figure and estimates the number of participants at just over 27 000.

Based on the leader boards, it is clear that some Folding@home participants contribute more to the project than others. Some participants download the software and run it on their home computers or lap-tops in the background, while others who are overclockers or hardware enthusiasts, contribute much more power through the modification of their machines. It is this latter group that can be considered the active participants in

Folding@home. However, it is difficult to ascertain what proportion of participants fall into the more passively-involved group, and who falls into the more active group. The fact that there are differences in estimates of the total number of participants makes any estimation of the proportion of the different ‘types’ of participant problematic.

41 Profile page of one of the highest ranked individual participants http://fah-web.stanford.edu/cgi- bin/main.py?qtype=userpage&username=Mtnduey. 42 ‘Kakaostats’ is widely consulted by ‘enthusiast’ participants of Folding@home. It was set up by a project participant and mines data measuring team and individual participant statistics produced by the Pande Group http://kakaostats.com/. [Accessed 5/8/14] 117

The Folding@home website only lists the top 100 teams, and the top 100 individual participants, so an estimate of the numbers of overclockers and hardware enthusiasts participating in Folding@home based on this data has been difficult. The Kakaostats site lists the top performing teams and the number of active individuals, although it is not always clear which teams include overclockers. However, it also provides the number of points generated per team member per day over the past seven days, and this can help to identify overclockers and enthusiasts. Overclockers can generate many thousands of points per day, while someone who contributes solely on a lap-top or PC (myself included) would take many weeks or months to generate the same amount of points.

Based on a close examination of the data on this website, and by counting participants who produce large numbers of points per day, I estimate that there are approximately

10 000 individuals participating in Folding@home who are overclockers and hardware enthusiasts. However, the discrepancy regarding the total number of participants remains.

4.2.5 Opportunities for interaction

The Folding@home forum is the main area where project participants can interact, either with other participants, or occasionally, with members of the project team. If a participant wants to post content on the Folding@home forum and interact with other participants then they must register separately, although anyone can read content.

Repeated visits and observations of this forum have shown that discussions are often highly technical with many of the threads focussing on software issues and hardware.

Figure 4.12 shows part of the Folding@home forum and illustrates the typically technical nature of the discussion threads.

118

Figure 4.12 A selection of discussion threads from the Folding@home forum43

Q&A about unsupported distros [software components] of Linux

FAH supplies a V7 client installer for Debian / Mint / Ubuntu / RedHat / CentOS /

Fedora. Installation on other distros may or may not be easy but if you can offer help to others, they would appreciate it.

There is also a ‘general discussion’ area in the forum (Figure 4.13) which contains discussion threads related to the science behind Folding@home, including links to media articles and coverage of the project. There is also an ‘anything goes’ section, although this also is related to technical aspects of the project, and discussions relating to software and hardware.

43 Screenshot taken on 24/03/14 119

Figure 4.13 General discussion area of Folding@home Forum

There are approximately 3-4 identifiable forum moderators, who are technically proficient, as well as a number of participants who contribute regularly to the forum.

Much of the discussion details advice (both from the moderators and from other participants) relating to installing Folding@home, software, and hardware issues.

Participants and moderators appear to be willing to help others get the most out of their technical involvement in the project. Discussions about the science are also similarly well-informed, with some forum participants showing an interest in the details of the science and in related areas of research. Links to new and related research are often shared by participants. The forum is the only area on the website in which there is interaction between participants. While there is a regular project blog written by

Professor Pande, this is a one-way transfer of news and updates, and feedback to any of these blogs is made via the Folding@home forum.

4.2.6 Participant experience

My involvement in Folding@home has been largely ‘passive’ in that I have simply downloaded and run the software. As I have no expertise in computing, membership of the Beta Testers is not an option, and I am not a hardware enthusiast or overclocker. I 120 have taken an interest in the science, read some of the publications and have also tried to encourage friends and family to participate. Dissemination of results and recruitment of new participants offer other ways to contribute for participants who aren’t computer hardware enthusiasts. Much of my experience as a participant has been exploring the overclocking community (of which I had no prior knowledge), and investigating the impact and reaction to a change in the project points system. Most of the interaction I had with other project participants was related to these two topics of interest.

Overclocking participants

The highly technical nature of the Folding@home forum made it feel somewhat inaccessible and I felt that the discussions were for ‘others’, although I was unsure who these ‘others’ might be. It wasn’t until I had obtained some feedback from the online survey that I was alerted to the presence of the community of overclockers. Given my unfamiliarity with this group and the lack of information available (searches were made of the academic literature, as well as more general online searches), I began to investigate further by observing and taking part in some of the discussion threads on overclocking forums and groups that had been mentioned by survey respondents. I registered with two of these forums: EVGA44 and HardOCP45.

Both of these forums give some background information relating to Folding@home and some of the science relating to the diseases investigated by the project. They link to updates and articles from the main Folding@home website, and actively try to recruit new team members. Both of these forums appear to be less formal or as stringently moderated as the Folding@home forum. Avatars are more widely used, and participants

44 EVGA is a company that manufactures games cards (GPUs), motherboards, and related accessories (http://www.evga.com/). Team members are customers of the company rather than employees. 45 HardOCP is an online magazine that offers news, reviews, and editorials that relate to hardware, software and overclocking (http://www.hardocp.com/). Team members are readers / followers of this site. 121 commonly post statistics related to their contribution (when they started, affiliations, points etc.). Participants use these forums for seeking and sharing advice about getting the most out of their machines. Some forum participants have also appeared to form online friendships with others in the group. While participants encourage each other, there is also an atmosphere of friendly competition and many use these forums for displaying and ‘bragging’ about their achievements (Figure 4.14)

Figure 4.14 Sharing achievements on the EVGA folding forum

This participant has just reached 5 million points.

This participant scored 1 383 630 points in a month.

This is a reference to a team achievement where they have produced more than 19 million points per day (PPD).

My observations of the enthusiasts’ forums also revealed more about the motivations for involvement in Folding@home, and helps to address RQ2 (what motivations initiate and sustain participation in online citizen science projects?). Both the EVGA and HardOCP forums have a discussion thread relating to why individuals fold. Many of the entries of

122 these discussion threads relate to the loss of friends and family to the diseases that

Folding@home researchers and some posts include poems, pictures and memorials.

Figure 4.15 shows some of the more typical posts from the HardOCP ‘Why we DC’ discussion thread.

Figure 4.15 ‘Why we DC’ from HardOCP forum

Through exploring some of the overclocking forums, I have found links to my own research survey with participants encouraging others to take part46. Some had promoted the survey as an opportunity for participants to provide important feedback to the project team and to “make their voice heard”. Such comments suggest that some participants

46 EVGA discussion thread relating to my survey and the summary of results: http://forums.evga.com/tm.aspx?m=1904069 Results of my survey shared at the Brony@home team forum : http://www.bronyathome.org/forums/showthread.php?tid=252&highlight=survey 123 within these groups value an opportunity to provide feedback to the project managers, and that these opportunities were not commonplace.

Changes in the Folding@home points scheme

For those with very powerful computers there is a separate points initiative called

‘BigAdv’ (‘big advanced folding’). By completing work units from this scheme, participants can generate many points and may also receive bonus points if the work units are turned around in a specific time-frame. This scheme therefore appeals to those who overclock and enables these participants to generate many millions of points. In December 2013, an announcement was made on the Folding@home forum by a member of the project team that there would be a change in the requirements for eligibility for BigAdv work units. This change would increase the processing capacity required, thus making many who currently participated in BigAdv ineligible. This change was greeted with some hostility, and the discussion thread relating to these changes on the Folding@home forum had over 700 comments. Discussion threads on other folding forums were similarly heated. On the EVGA folding forum there were several calls to begin some sort of ‘strike’ or power outage in response to the announcement (Figure 4.16).

The first comment (1) makes a reference to a recent purchase of hardware that would have been used to generate many points through the BigAdv scheme, but which may now prove to be useless and may be sold. The second (2) participant suggests that

Folding@home participants go on strike to protest the changes in points scheme. The third comment (3) outlines the investment that this participant has made in terms of hardware and electricity, and expresses some doubt whether any scientific advances have been made as a result of the efforts of participants. Members of the Folding@home team are called “ungrateful”, for the “millions of dollars of work done for free”.

124

Figure 4.16 Feedback relating to changes in BigAdv points scheme from the EVGA forum

The reaction to this announcement has shed further light on the motivation of this community of Folding@home participants, and the importance of points as a reward for their contribution. Some comments also demonstrate that participants in these projects

(especially those that make a substantial contribution), want to be valued and appreciated for their efforts by the project organisers. Since this announcement was made, some folders in the EVGA team stated on the forum that they were going to devote their computing resources to other distributed computing projects and the number of folders on the EVGA team decreased slightly in the months following the change in BigAdv.

125

Less than a month after the proposed changes to BigAdv were announced, a forum post and blog update47 appeared stating that the proposed changes were to be revised so that that fewer participants would be affected, and that there would be a delay in the time that the changes to be BigAdv would be implemented. In this message were comments relating to communication with participants, and that greater input from donors would be sought with regard to future changes in the project. This post shortly followed an announcement from the project team that a community relations manager was to be appointed48.

Participants as ‘donors’

The Folding@home project team refer to participants as ‘donors’. This is in contrast to other online citizen science projects such as Planet Hunters, where the project team refer to participants as ‘collaborators’. Furthermore, there is little, if any, acknowledgement of the efforts of the Folding@home community in the scientific publications that arise from the output of the project. Again this is in sharp contrast to both Planet Hunters and Foldit where the efforts of participating citizen scientists are acknowledged, and where those who make a significant contribution can be cited as co-authors. This observation is of interest in relation to RQ 6 (how do participants perceive their role in the project?), and the attitudes of the project teams towards participants may be an important factor in how participants view their own contribution. This issue will be considered in greater detail in relation to the survey and interview feedback and in Chapter Eight.

47Blog post concerning recent BigAdv changes and how they have been revised: https://folding.stanford.edu/home/revised-plans-for-bigadv-ba-experiment/. 48Blog post with item relating to the appointment of a community relations manager: https://folding.stanford.edu/home/happy-holidays-looking-back-at-2013-and-forward-to-2014/. 126

4.2.7 Folding@home summary

Folding@home is a powerful distributed computing project that has resulted in the production of over 100 scientific publications. Many participants (although the actual number or proportion is difficult to ascertain) simply run the project software and do not get involved any further in the project. However, there is a community of participants

(estimated to number 10 000) who make a substantial contribution to the project through their involvement in overclocking. These participants interact on the Folding@home forum as well as on a number of other online forums and communities dedicated to overclocking. This interaction involves some co-operation and collaboration between participants as they work to address technical issues relating to their overclocking activities. These observations have helped to address RQ4 (How and why do project participants interact online?). Some overclockers are motivated to participate because they have had some personal experience of the diseases that Folding@home seeks to understand and / or they enjoy competing with other participants. Changes to the points system have also demonstrated that importance of points as a motivator for sustained participation and the importance of being acknowledged and appreciated for their contribution. Thus helping to address RQ2 (What motivations initiate and sustain participation in online citizen science projects?).

127

4.3 Planet Hunters (www.planethunters.org)

Planet Hunters is a distributed thinking project which is part of the ‘Zooniverse’ group of online citizen science projects. The Zooniverse began in 2007 with the launch of its first project Galaxy Zoo, which enlisted the help of volunteers to classify images of galaxies taken by the Sloan Digital Sky Survey (Lintott et al., 2008). By August 2014, the

Zooniverse was hosting twenty-three projects from a range of disciplines and involving teams of scientists and developers based at a number of UK and American universities.

Over one million individuals have registered to take part in Zooniverse projects since 2007

(Simpson, 2014a). Planet Hunters was launched in December 2010 and is an astronomy- based project that was set up by scientists from Yale University and Oxford University. It enlists the help of citizen scientists to search for planets outside of the solar system orbiting distant stars (also known as exoplanets).

4.3.1 Underlying science and the project task

Planet Hunters involves the analysis of data collected by the NASA Kepler space observatory (Schwamb et al., 2014). The Kepler spacecraft monitors approximately

160 000 stars, and continually records their brightness taking a measurement every half an hour. Small periodic reductions in the brightness of a star can be the result of a planet crossing in front of it, and is known as a transit (Bhattacharkee, 2013). Participants in

Planet Hunters are given graphical data, referred to as light curves, for a star in the Kepler field. These illustrate variations in the star’s brightness over a given period (Figure 4.17).

128

Figure 4.17: Light curve for one of the stars in the Kepler field. Each point represents a measurement of the light intensity of that star at a given point in time. The first task that participants undertake (1) is to answer a question about the shape of that light curve.

After the participant has made this initial classification of the star, their next (and most important) task is to look for evidence of planetary transits which manifest themselves as dips in the brightness as a planet passes in front of the star. These appear on the light graph as points below the main body of the light curve (Figure 4.18). This example provided in the project tutorial, illustrates a clear example of a planetary transit.

However, most of the light curves given to participants to classify are not as clear and contain considerably more ‘noise’ than the example below. Each light curve is looked at by 5-10 different participants.

129

Figure 4.18: Planetary transit. Evidence of a transit is observed a dip in the recorded brightness of the star. These are marked with click-and-drag boxes (in blue). This star can be discussed by the participant, or they can move on to the next light curve.

These tasks are carried out by individual citizen scientists, and there are no teams on

Planet Hunters. Individuals create a username at the time of registering and they can be used on any Zooniverse project. No personal information is required, and a Planet Hunter user profile only displays the discussions or ‘Talk’ comments have been made. These profiles can also be used to send a private message to another participant.

The data from the Kepler mission has been publicly archived, and is openly available for analysis49. In May 2013, the Kepler spacecraft suffered a malfunction which has brought its extended mission to a close. However, there is ample data that has already been collected to keep the Planet Hunters project going until 2016 (Schwamb, 2013). In

49 Active Kepler dataset: http://kepler.nasa.gov/science/ForScientists/dataarchive/. 130 addition to human analysis of the Kepler data, an algorithm was developed by NASA scientists that also analyses light curves. This algorithm has led to the discovery of hundreds of planets, some of which have been confirmed by Planet Hunters participants.

The work of citizen scientists on Planet Hunters complements this automated analysis, and human observation has highlighted a number of unique systems that have been overlooked by computers (Schwamb et al., 2013).

There have been seven publications in peer-reviewed journals based upon the work of citizen scientists involved in Planet Hunters50. This includes the discovery of two new extrasolar planets by project participants – PH 1b and PH 2b. Links to a list of all Planet

Hunters participants have appeared in the acknowledgements of these publications51, and the individual participants who have been involved in the discovery of the new planets have appeared as co-authors (Lintott et al., 2013, Schwamb et al., 2013, Wang et al., 2013, Schmitt et al., 2014).

4.3.2 Active playing population

Just over 160 000 people have registered with Planet Hunters (Schwamb, 2014a) and it is one of the more popular projects on the Zooniverse accounting for approximately 27% of all activity on the site (Simpson 2013). However, participants don’t always spend a great deal of time classifying light curves, and an individual visit to the site lasts, on average, just over 11 minutes (Simpson, 2013).

Of the many thousands of registered participants, the number that contributes regularly is quite small. A snapshot of participant activity provided by the Zooniverse Team in May

2013 (Raddick, 2013) indicated that there were 160 671 registered participants in Planet

50 List of all publications based on Zooniverse projects https://www.zooniverse.org/publications. [Accessed 7/8/14] 51List of all individuals who have contributed to Planet Hunters http://www.planethunters.org/authors. 131

Hunters. Of these registered users, 114 929 (71.5%) had not marked any transits on the light curves, and had just completed (or just arrived at) the first task of identifying the shape of the light curve. Of the 45 742 people who had completed the second task and looked for transits, nearly half (21 620) had analysed fewer than 10 light curves. Only

4 500 participants had analysed over 200 light curves, and had thus carried out the majority of the work since the project began in December 2011. Even within this group of more active participants, the data has also revealed that a small number of individuals have done a disproportionate amount of the work. Forty individuals have classified over

10 000 light curves each, with one individual classifying over 100 000 light curves.

The number of participants who are currently active in Planet Hunters can be estimated from observations of the appearance of participant usernames on the discussion boards, comments made to project blogs, and by monitoring the number of players playing at regular intervals (all registered players currently taking part in Planet Hunters are listed at the bottom of the screen). Monitoring the number of current participants is of particular importance, as not all those who take part in Planet Hunters interact with other participants, although they may read posts from other participants. From these observations, it is estimated that approximately 300 individuals actively participate in

Planet Hunters. These participants are classifying light curves at least once a week. Of this group, a small number have become more involved in the project. These individuals contribute at least four times per week, participate regularly in the online discussions and make use of the opportunities to communicate with project scientists. They are easily recognised by their usernames. A small number analyse the raw Kepler data, and take a greater interest in the science relating to exoplanets. One of the project scientists interviewed, estimates that there are 20 individuals in this highly active ‘core’ group.

132

This pattern of participation in which the bulk of the work is carried out by a relatively small number of participants is a feature of other Zooniverse projects (Ponciano et al.2014, Tinati et al., 2014), and is illustrated by the following graphic (Figure 4.19).

Figure 4.19 Pattern of participation in Planet Hunters.

The figure contains a box for each participant. The area of the box is proportional to the contribution of that participant. One large box (1) represents the contribution of individuals who were not logged on or registered with the project. 52

Recent research on the patterns of participation in two other Zooniverse projects (Galaxy

Zoo and the Milky Way Project) describes participants as either transient (those who execute tasks on only one visit) or regular (those who return at least one more day)

52 This figure was kindly provided by Dr Grant Miller from the Zooniverse team. A similar graphic has been produced for Old Weather, another Zooniverse project. http://blog.oldweather.org/2012/09/05/theres-a- green-one-and-a-pink-one-and-a-blue-one-and-a-yellow-one/. 133

(Ponciano et al., 2014). This study estimates that 67% of the 100 000 volunteers studied over a two-year period are transient.

4.3.3 Interaction between participants

Participants have an opportunity to interact with each other, and with the project scientists on the Planet Hunters website. There are three separate features that allow this interaction: the ‘Talk’ function; the discussion boards; and the project blog.

The Talk function53 is used by participants to draw attention to light curves of interest.

After analysing a light curve, the participant is asked whether they would like to talk about it. If they do, then they can post a message (which has a limit of 140 characters) which then appears with the image of the light curve in the Talk section under a list entitled ‘Recent Objects’. Other participants may look at these objects and comment, perhaps checking to see if they see any evidence of a transit, or if the object shows some kind of unexpected feature. Images can be grouped into ‘collections’ and project scientists can monitor these for any patterns of interest.

In addition to the Talk feature, participants also have the option to communicate with each other on the discussion boards54 (these do not impose a character limit). The discussion boards are divided into ‘help’, ‘the science’ and ‘chat’ and provide a forum where participants can learn more about the project, and are able to discuss items of interest in more detail than in the Talk function. Both citizen scientists and members of the project team use these areas and they are moderated by three different citizen scientist volunteers. Discussions are generally related to the project and the associated science, even in the ‘chat’ section. A recent study investigating how Planet Hunter

53 Link to Talk area of Planet Hunters. http://talk.planethunters.org/ 54 Link to the ‘chat’ discussion board. http://talk.planethunters.org/chat 134 participants learn about the project task, found that the Talk function was used much more widely that the discussion boards. There were almost 330 000 contributions to Talk compared with 17 000 contributions to the discussion board (Mugar et al., 2014).

One or two members of the scientific team communicate with the project participants through the Planet Hunters Blog which contains news and updates about the project, background information about the science, and details relating to the publication of research or presentations at academic meetings. Citizen scientist volunteers can comment on these posts. Occasionally there are guest blogs from postgraduate students, and moderators of the discussion boards. Zooniverse projects, including Planet Hunters, are establishing a presence on social media sites such as Twitter (over 8 600 followers) and Facebook (over 320 000 ‘likes’)55.

4.3.4 Participant experience

I first heard about the Zooniverse in 2009 while undertaking an MSc project in astronomy outreach (Curtis, 2013) and became a participant in their first project (Galaxy Zoo) for a couple of months. In January 2011, a new Zooniverse project, Planet Hunters, was promoted on the BBC ‘Stargazing Live’ programme. This was of interest to me and I participated occasionally over the next few weeks. I did not visit the Zooniverse again until October 2011 at the beginning of my PhD research where I found that the number of projects had increased significantly. Previous knowledge of the Zooniverse and of Planet

Hunters has helped to establish my interest in online citizen science projects, and influenced my choice of Planet Hunters as a case study in this research.

55 As of August 2014. 135

Learning and carrying out the project task

The task of classifying light curves on Planet Hunters differs from that of other Zooniverse astronomy projects as the participant is not looking at an astronomical image, but at graphical data. The task can be demanding as it is not immediately obvious whether there is evidence of a planetary transit, and some of the light curves require close examination. There is a short online tutorial, although this is based on an example where there are obvious signs of a planetary transit. Many of the light curves in the Kepler database that I have been given to analyse have had a substantial amount of ‘noise’. One source of frustration while learning the project task was that there is no feedback given to participants if they have made an incorrect classification, or indeed, a correct one. Space

Warps, another Zooniverse project has ‘dummy’ or constructed examples of the items of interest (in this project, it is visual evidence of gravitational lenses) that are periodically given to participants to classify. The participant is then told if these were correctly or incorrectly assessed, so providing some feedback and a reminder of what it is they are actually looking for. I felt that such a facility would be useful in Planet Hunters, and would help to sustain participation.

However, as a participant, the fact that each light curve represents an actual star (with an alpha-numeric reference), made the task feel authentic, as did the fact that this data was derived from an ongoing NASA project. Both of these factors helped to motivate me while learning the task, and carrying it out. The Planet Hunters task itself is somewhat monotonous and I did not usually spend more than 10 minutes classifying light curves per visit. As of August 2014, I have classified over 500 stars. I often visited other Zooniverse projects in one session if I had the time, and spent a similar length classifying on other projects. According to members of the Zooniverse team who were interviewed, many

136 participants are active on a number of Zooniverse projects. The amount of time devoted to each is recorded, and total contribution can be displayed on a personal ‘ribbon’ of participation (Figure 4.20 illustrates my participation on Zooniverse projects).

While I was a regular contributor, I seldom took part in anything apart from classifying light curves. I have not examined the raw Kepler data and only occasionally took part in online discussions or used the Talk function. There is little requirement to go beyond the main project task, and there are not many alternative tasks available apart from forum moderator. Despite the presence of the Talk function and the discussion boards, my participation in Planet Hunters was mainly a solitary pursuit.

Figure 4.20 Participation ‘ribbon’ for the Zooniverse

1 400 total classifications have been made in 16 projects

projects

Most activity has been on Planet Hunters

137

Observing interaction on Planet Hunters

Much of the content of Planet Hunter interactions was technical in nature and was related to the project task. In all communication spaces interaction was dominated by the same small group of participants (I observed the same 10-15 participants). There are a small number of discussion threads that are slightly more personal nature. For example there is one entitled ‘who are the Planet Hunters?’ in which a small number of participants have introduced themselves and given a bit of background information such as where they’re from, occupation, and what brought them to the project. For many, their participation was motivated by an interest in astronomy, and the opportunity to make a new discovery. This thread is over two years old however, and there have been no new posts for many months. Of the team of project scientists, only one is visible to any great extent. She writes the majority of the blog posts, and responds to items raised on Talk and on the discussion boards.

My interaction with other participants in Planet Hunters has been relatively limited. I have seldom used the Planet Hunters Talk function, as I felt that very few of the light curves warranted further discussions and I have rarely observed evidence of planetary transits. On the few occasions where I have posted a light curve on Talk that I was interested in, I received no feedback or comments from other participants. The lack of any response has been a disincentive for further use. The fact that Talk comments are limited to 140 characters also made this feature less appealing to me, and I have preferred to use the discussion board or the project blog for finding out what is going on within the project.

While much of the discussion on the forum can get quite detailed and technical, I have taken part in several of the discussion threads, including the ‘Who are the Planet

138

Hunters?’ thread. However, I received no response or feedback from any of the other individuals who had contributed to this conversation. I used this thread as an opportunity to talk a bit about my research (as well as make my presence as a researcher known) and why I was interested in Planet Hunters. I also posted a link to my survey on the discussion board, and while it attracted a number of respondents, in generated no discussion or further comment on the website. While there is evidence of a small group of active contributors, they appear to be distant and hard to engage with unless one has a greater level of technical interest in the project. Interaction in Planet Hunters appears to be limited to this group of participants.

On several occasions, the Zooniverse team have organised meet-ups in the UK for participants of Zooniverse projects, and in June 2013 I attended the ‘Zoocon’ event at the

University of Oxford. Several of the Zooniverse project scientists were there and presented project updates, and I was able to meet some of the core participants from a number of the Zooniverse projects. However, it did not appear that any of core group from Planet Hunters had attended (the forum moderators and two individuals who have appeared as co-authors on Planet Hunters papers are based in the US). Many of those who attended had been with the Zooniverse for a number of years and knew each other through their involvement in Galaxy Zoo. These individuals had established friendships with each other, as well as with some of the project scientists, and several of the attendees were forum moderators. This meet-up took place just after I had launched my survey, and I was able to talk to several participants about my research. At a number of points during this event, project scientists expressed their gratitude to participants for their contributions, and one stated that the event was a way to keep participants involved

139 and make them feel valued, thus demonstrating the importance that recognition may play in sustaining participation in these projects.

4.3.5 Planet Hunters summary

Planet Hunters is a distributed thinking project which has resulted in a number of important discoveries of exoplanets. The project task is relatively straightforward and can be carried out during short visits to the project website. Participant-observation has shown that taking part in Planet Hunters can be relatively solitary. The task is completed alone, and there are no organised teams. While there are 160 000 registered participants, most have completed very few (if any) classifications of the light curves. The active playing community is estimated to be 300, with an active core group of participants of approximately 20. Interaction between Planet Hunters participants mainly involves this core group and they are frequently observed on the Talk function and discussion boards. They also interact with one of the project scientists through the discussion boards and via the project blog. These observations have helped to address RQ 4 (How and why do participants interact online?). In some online discussion, participants have stated that an interest in astronomy and the desire to make a discovery have motivated their participation to take part in Planet Hunters. Other motivations that initiate and sustain participation (RQ 2) have been illuminated by the survey and interview feedback, and will be explored in further detail in Chapter Seven.

4.4 Conclusion

RQ 1 asks who participates in online citizen science projects? Observations of these projects have revealed that they all have a much smaller number of active contributors when compared to the total number of registered participants. While many individuals show an initial interest, not all will persevere with learning the project task or completing

140 more than a few ‘clicks’ or classifications (Ponciano et al., 2014). However, the definition of an active participant varies between projects, and is related to the project task. For example, Foldit has a greater level of difficulty and so requires a greater level of time commitment than Planet Hunters. In Folding@home, an active participant is one that makes modifications to their computers in order to increase their contribution (and points per day), rather than an individual who simply downloads and runs the software.

In both Foldit and Planet Hunters, there is a committed core group of individuals who play a crucial role(s) in these projects. Not only do these individuals complete much of the main project task, but they may also take up other roles such as forum moderators, team co-ordinators, translators of content, and authors of other project-related content such as wikis and FAQs. These participants are equivalent to the ‘leader’ group as identified by

Preece and Schneiderman (2009) in their ‘reader-to-leader’ Framework. In

Folding@home, more active participants can be found within the Beta Testers group and the community of overclockers and hardware enthusiasts.

This research ultimately focuses on the active participants, and a smaller number of core participants in each of the three projects. These individuals are important as they help to sustain these projects. Their continued involvement in the projects also makes them easier to observe, and easier to contact and recruit for surveys and interviews.

Participant observation has also shed some light on what motivates participation in online citizen science projects for the citizen scientist volunteers (RQ2). Interest in the underlying science, the possibility of making a discovery, a points-based reward, and friendly competition have been highlighted in forum discussion threads within these projects. RQ 4 seeks to address how and why project participants interact online, and observations have shown that each project has a number of available opportunities to

141 interact with other citizen scientists, and with project scientists and developers. In the case of Foldit and Planet Hunters, interaction with project team members is more likely to involve core participants.

In addition to the main project task, there are other ways in which citizen scientists can become involved in a project. Sometimes these other tasks require a certain skill (e.g. members of the Beta Testers in Folding@home require programming skills), or a degree of commitment to a project (e.g. forum moderators must spend time observing discussions). These observations help to further characterise contribution (RQ5) beyond the overall level of participation by identifying other project-specific roles.

Through my experiences as participant-observer, I have begun to address some of the research questions, and to create a context within which to consider the findings of the surveys and interviews in Chapters Five, Six and Seven. However, this participant- observation data stream will be further considered alongside the feedback from the surveys and interviews in Chapter Eight, where the findings from all three projects will be assimilated, compared and analysed.

142

Chapter 5: Foldit results This chapter is the first of the three results chapters and it presents the findings from the

Foldit player’s survey (see Appendix B) and a thematic analysis of the interviews with 10 players and three members of the project team.

5.1 The online survey results

In total, 37 players responded to the survey. While this is a small number it represents

12-18 % of the active playing community (see Chapter Four, Section 4.1.3). However, their views may not be representative of all Foldit players, and this must be taken into consideration when interpreting the results of the survey.

Survey results have been broken down into four main areas of interest that have direct relevance to the research questions: demographic characteristics (RQ 1); patterns of participation (RQ 5); motivation and reward (RQ 2), and interaction with other players

(RQ 4). Not every respondent answered every question. For the results of each individual survey question presented, n=37 unless otherwise stated.

5.1.1 Demographic characteristics

The majority of respondents were male (78%) and most (68%) were aged over 40. The over-representation of men has been observed in other online citizen projects (Holohan and Garg, 2005, Krebs, 2010, Raddick et al., 2013, World Community Grid, 2013, Estrada et al., 2013). Fourteen (38%) of the respondents were from the US, and 18 (49%) were based in European countries. The rest were from Canada, New Zealand, and Thailand.

Figure 5.1 illustrates that this group of respondents is very well educated. Seventeen respondents (46%) had an undergraduate degree, and nine (24%) had a postgraduate

143 degree. Very few (6 individuals) had only the equivalent of a high school education, although 2 of this sub-group of respondents were still attending high school.

Figure 5.1 Highest educational level attained

high school / A levels

15% 3% 23% HND / technical college diploma some undergraduate 12% course completed undergraduate degree 47% postgraduate degree

The majority of those that had a university education (both undergraduate and postgraduate) were qualified in a STEM subject (science, technology, engineering and mathematics) (see Figure 5.2).

Figure 5.2 Highest educational qualification in a STEM subject

3% no formal 5% qualification 13% high school / A levels 22% 3% HND / technical college diploma 11% some undergraduate course completed 43% undergraduate degree postgraduate degree

online / self-taught

144

The occupations of respondents are listed in Table 5. 1. There is a high representation of those in the IT sector. One respondent reports their occupation as biologist.

Table 5.1 Occupation or profession of respondents (previous occupation given if retired or unemployed)

IT – related 13 Student 5 Economist 2 Gardener 2 Librarian 2 Biologist 1 Teacher 1 Conservationist 1 Clerical worker 1 Factory worker 1 Car mechanic 1 Inventor 1 Sales 1 Sports instructor 1 Publishing 1 Translator 1 Not given 2

5.1.2 Background interest in science and participation in related activities

In addition to participation in Foldit, I was interested in whether respondents regularly play any other computer games. A number of print and online media articles written about Foldit make much of the fact that ‘gamers’ are helping to solve scientific problems

(Bourzac, 2008, Gross, 2012), and the Foldit team appear to highlight this packaging of

Foldit as a game when promoting it to external audiences (Burke 2012, Cossins, 2013).

Twenty-two of the respondents (59%) stated that they did not play any other games apart from Foldit.

145

Whether respondents had a wider interest in science was explored by asking them if they had taken part in other science-related activities during the previous year, this included participation in other citizen science projects.

Nearly half of respondents (18) had taken part in other citizen science projects and Figure

5.3 illustrates which ones they had participated in during the past year. Only two respondents stated that they were still currently involved with other citizen science projects.

Figure 5.3 Participation in other citizen science projects (the number of participants participating in each project is at the top of the bar)

4 3

1 1 1

Figure 5.4 illustrates that the majority of respondents (35 respondents) had participated in a number of other science-related activities during the previous year. Only two respondents reported not taking part in any. Watching scientific television programmes and reading popular science books were the most common activities listed, followed by listening to scientific radio programmes and visiting science centres or museums. 146

Figure 5.4 Participation in science-related activities during the past year

22 18 15

10 8 5 3 2 2 2 1 1

Perhaps the most notable feature of the Foldit demographic data is how well educated the respondents are, and the fact that the majority have a tertiary-level education in a science, technology or engineering.

5.1.3 Patterns of participation

Just over a quarter of respondents (10) had been playing Foldit for less than six months

(see Figure 5.5). Seven respondents (19%) had been playing for over three years. This sub-group of long-term players contains some (approximately seven) of the Foldit core group, the small group of participants who are highly committed to the game, and have reached a high level of proficiency (see Section 4.1.3 in Chapter Four).

147

Figure 5.5 Length of time respondents have been playing Foldit

19% < 6 months 27% 6-12 months 8% 12-18 months 18 months - 2 years 11% 2-3 years > 3 years 5% 30%

Only six respondents (16%) described themselves as ‘beginner’ level, while 16 (43%) described themselves as ‘intermediate’ and 15 (41%) considered themselves to be

‘advanced’ players. These levels were not based on any particular criteria, but gauged how the respondent viewed their own standing within the game. Thus, this sample is skewed towards those who consider themselves to be relatively proficient in the game.

Foldit players have the option to join a team. And among this group of respondents, 23 individuals (62%) belonged to a team, 11 (30%) defined themselves as ‘solo’ players, while

3 (8%) stated that played both within a team and as solo players.

Figure 5.6 illustrates how long (on average) survey respondents report playing Foldit per week. Many within this group of respondents spend a considerable amount of time participating in Foldit.

148

Figure 5.6 Number of hours per week (on average) respondents play Foldit

< 2 hours 2-4 hours 5-10 hours 11-15 hours 15 hours or more

3% 8%

49% 24%

16%

Almost half of respondents (18) are playing Foldit for 15 hours or more a week. Only 11% of respondents (4 individuals) are playing for fewer than 4 hours per week. Given that the majority of respondents are in full-time employment or education, this could be considered a relatively high level of commitment to the project, particularly if this has taken place over many months or years.

Question 6 asked respondents about their participation in the Foldit online community via the internet relay chat and the website forum. Thirty-one respondents (84%) stated that they interact with other players, with nearly a third of respondents (12 individuals) participating for more than 5 hours a week on average (see Figure 5.7). Of this group of respondents, 3 individuals specified that they tend to read posts and content rather than contribute input of their own.

149

Figure 5.7 Number of hours per week respondents participate in online discussions (n=31)

<1 hour 1-2 hours 2-3 hours 3-4 hours 5+ hours

32% 33%

3% 3% 29%

Most of the respondents specified that this interaction takes place during the game within the various chat windows. Therefore, this aspect of participation does not necessarily add any more time to their total project participation. However, some participants take the time to post in the online forum, or take part in one of the scientist’s or developer’s chats.

5.1.4 Motivation and reward

Respondents were asked why they decided to play Foldit. This was to address the first part of RQ2 concerning motivations that initiate participation in an online citizen science project. The responses to this question are illustrated in Figure 5.8. Practically all of the respondents gave more than one reason for trying the game although a few motivations were dominant.

150

Figure 5.8 Why respondents decided to participate in Foldit

10 8

4 3 3 2 2 2 2 1 1 1

Clearly, one of the key motivations for beginning to play Foldit is the opportunity to make a contribution – to science, to medicine, to the development of new drugs. Twenty-two respondents (59%) stated that this was one of the reasons that they began playing Foldit.

Thirteen respondents (35%) are motivated by a background interest in science, which may be related to the fact that many have undergraduate and postgraduate qualifications in science or technology-related subjects, and take part in a number of science related activities in addition to Foldit.

The intellectual challenge attracted 10 respondents (27%), while eight (22%) tried Foldit after their curiosity was aroused. Only a small number of participants (3) were attracted

151 to Foldit as an opportunity to learn something new, while the same number of respondents was drawn to Foldit because it was a computer game.

Respondents were asked what they liked best about Foldit. This question was used to elucidate the motivations that sustain participation in the project. The responses to this question are illustrated in Figure 5.9.

Figure 5.9 What respondents like best about Foldit

14 13

9 8 8

3 3 2

Again, making a contribution to research is important for 14 respondents (38%) and may motivate their continuing participation in Foldit. The interaction with others and sense of ‘community’ was mentioned by 13 respondents (35%). Learning something new and developing new skills were important for approximately a quarter (9) of the respondents.

Only three respondents mentioned specifically that they thought Foldit was fun to play, although the enjoyment of playing can be inferred from other comments – particularly

152 those that refer to the community aspect of the Foldit. Once involved in the game, the importance of the community aspect of participation becomes more apparent. Perhaps working closely with others toward a useful scientific goal lends an air of seriousness to

Foldit, and may explain why there are fewer comments referring to the ‘fun’ in the game.

Research into computer games has generally found that players are strongly motivated to play if they find the game enjoyable or fun in some way (Williams et al., 2008, Yee, 2006,

Yee, 2007).

Question 17 explored the concept of rewards in more detail and asked respondents if they thought Foldit players should be rewarded for playing, and if so, what would be the most appropriate way. Overall, 35 respondents provided an answer to this question. Of this group, 23 (66%) stated that they did not think that Foldit players should be offered an extra incentive. The view that rewards could harm the cooperation within the Foldit community was expressed by several respondents.

“More concrete rewards would probably increase the incentive, but would likely also reduce cooperation, so it’s a double-edged sword.”

The potential commercial spin-offs of Foldit work and who should benefit from this was raised by three respondents, mainly among the nine who had stated that they thought

Foldit players should receive a specific reward for their time and effort. Three other respondents would like more public (and specific) recognition.

“Yes. At the very least by naming each individual who, by playing the game and coming up with a solution, contributed to a breakthrough, in publications. Have these people share in patent revenues and other sources of income.”

153

The responses to this question reinforce the observation that the respondents are attracted to the game in order to make some sort of contribution to science, or the advancement of scientific knowledge (and the societal and personal benefits that they may bring), rather than for any tangible rewards. The importance of making a contribution has been observed in previous studies exploring motivation to participate in online citizen science projects (Holohan and Garg, 2005, Raddick et al., 2010, Krebs, 2010,

World Community Grid, 2013). The view (of most respondents) that a financial incentive of some sort would have a negative impact on the Foldit community and reduce cooperation indicates that these players are focussed upon the long-term outputs of the game, and the potential societal benefits of the results.

5.1.5 Interaction with other participants

Question 16 asked players to describe their interactions with other players in Foldit. Of the 29 players who responded to this question, 25 provided responses that were interpreted as positive and they described interactions as friendly, supportive, helpful, enjoyable – even ‘world changing’.

“I count my team mates as friends and treat them as such. Being in a small team we have got to know each other very well. In the wider community other players are always treated with respect and helped where possible. Generally everyone is very good and we tend to get on well with each other.”

“So far it has been pretty good. I feel heard and respected for my interactions for the most part. With so many people and so many cultural back grounds it's not always easy, but it can be so much fun”.

“Sociable, friendly, warm, supportive, intelligent.”

154

These respondents refer to talking “off-topic” with fellow team members, and discussing more personal topics. Foldit players come from a variety of countries and cultural backgrounds, which has to be taken into consideration when participants interact with one another – especially on global chat. Five of the respondents stated that they didn’t really interact very much, mainly because they are new to Foldit, or don’t have the time.

Discussions between Foldit players can be quite technical at times, with players using scientific terms relating to the science of protein folding, or technical terms relating to coding recipes. Therefore, new players may not feel confident enough to make a contribution for some time.

In total, only four of the respondents gave answers that could be interpreted as negative.

One respondent described their interactions with other players as “insubstantial”, while there are privacy concerns for one respondent.

“Privacy concerns keep me from sharing much personal info on the Foldit site.”

Despite this small number of more negative comments, interactions reported by these

Foldit players appear to be very positive and supportive, and friendships have been formed between participants, particularly those within the same team.

As outlined in Chapter Three (Section 3.4.1) a quantitative analysis of the survey results was attempted, but the number of respondents was too small to permit any meaningful statistical tests.

5.2 Interview feedback

The results of the online survey informed the interview schedule by highlighting areas that required further investigation including motivation, the skills required to play Foldit and the role of the project team. The results of the interviews have been subjected to a

155 thematic analysis using the approach of Braun and Clarke (2006). This approach has been described in greater detail in Chapter Three (Section 3.4.2). The major themes will be presented for both the interviews with Foldit players (n=10) and with members of the project team (n=3).

5.2.1 Interviews with Foldit players

Emerging themes from the player interviews were identified through a process of inductive analysis. Nine of the interviews were carried out via email, and one via Skype.

Table 5.2 details the emerging themes from the interviews with the citizen scientists56.

Table 5.2 Emerging themes from interviews with Foldit players

The ‘right stuff’ The essential qualities of a good Foldit Player, and how the high barriers to participation have led to the establishment of a small playing population. Qualities considered to be the ‘right stuff’ have a direct bearing on other aspects of player experience, and therefore, on other emerging themes. Learning how to play Foldit Player experiences and strategies involved with learning the basics of Foldit, and how they develop their own unique approach to playing the game. How participants have adapted to the game as it has evolved since its launch. The Foldit community The type and strength of online community interaction and some of the associated issues. The relationship between the players and the the Foldit team of scientists and developers. Why I play Foldit Factors that have motivated players to begin playing and remain commited to the game. Aspects of the game that are enjoyable for players. Science as a game Some of the issues involved in repackaging a scientific research problem as an online game, and how effective this approach is from the players perspective.

The ‘right stuff’

This is one of the most recurrent themes that emerged from the data as it appears to have a bearing upon several of the other themes. It relates to the concept of an essential

56 Interview questions are listed in Appendix D. 156 skill set that is required to be a good Foldit player. One respondent referred to this as the

“right stuff”.

This wider skill set can be described as comprising of intellectual attributes such problem- solving skills, intellectual curiosity, and pattern recognition skills. It may also comprise of science-related skills including a background interest in the science, or a formal qualification in a STEM subject. However, the attributes most commonly referred to by interviewees were personal or character attributes such as perseverance, obsessiveness, patience, ‘people skills’, determination, and dedication. Several of the interviewed players stated that it is a combination of these qualities that makes a successful Foldit player.

“By far the two attributes that help Foldit players are an obsessive personality and scientific inquisitiveness. With these, a player will eventually figure out strategies that help him to do well.” (Respondent FD1)

Respondents also spoke of a “state of mind” or an “attitude” that was required by successful Foldit players.

“More than knowledge or skill, the game requires a state of mind. The game requires patience because you can play for a long time without result of any kind.” (FD5)

This attitude appears to be a combination of character traits such as patience, perseverance and a degree of obsessiveness. Approximately half of these interviewees are within the Foldit ‘core’ community, and these individuals are playing for several hours every day, as well as managing Foldit teams or moderating player discussions.

157

Even among the committed Foldit players interviewed, there is an acknowledgement that

Foldit is a very difficult game to play, it can take up a lot of time (and financial resources), and some of the puzzles can be extremely challenging.

“…participating in Foldit not only means having your computer running, but it’s really working. I spent hours constructing molecules…” (FD2)

One respondent spoke about the misrepresentation of Foldit by news media, and the fact that new players had unrealistic expectations of the game. Ultimately, they weren’t prepared for the level of difficulty and commitment that authentic scientific research entails.

“Many new players see this game through other media, see the words ‘play for science’ and jump in. They start the tutorials, not really paying attention and asking for help instead of actually reading what is presented to them, and then ask ‘am I curing cancer or

AIDS?’. Then they realize it isn’t as glamorous as they thought and go back to their Call of

Duty or Left for Dead. There are a few who persevere though, and start the road to actually contributing to the ‘collective knowledge’.” (FD9)

The responses from the interviewees suggests that the Foldit project team have to cast their net far and wide in order to attract this small number of people who have the right background interests, technical skills and personality attributes. Correspondingly, those with these attributes may be more motivated to play Foldit.

Learning how to play Foldit

In a sense, the learning process is a test or a filter that determines whether an individual has the ‘right stuff’ to carry on playing the game, and players make decisions during the learning process that affect the way they subsequently play the game and interact with

158 other players. It is recommended by the Foldit team that all new players complete the set of tutorial puzzles. In addition to the tutorial puzzles, there is a player-constructed wiki that contains advice for new players as well as simulations of the tutorial puzzles. If one gets stuck, then they can view the solution via the wiki. It was generally acknowledged by the players interviewed (even those that have gone on to become very highly ranked players) that learning how to play can be difficult and time-consuming.

“Learning to play Foldit was quite agonizing…it took me 17 days to complete the tutorials on my outdated Windows XP workstation.” (FD4)

There is also a sense of people learning to play through trial and error and by “fumbling around”.

“I learnt by taking the long road; reading the bubbles; making mistakes; resetting, and understanding each tutorial. As for the main puzzles, it was, once again, the long road. I looked at the wiki but I mainly just got on with it.” (FD9)

While learning by doing, and finding one’s own way through the tutorial puzzles is important, there is also a sense of the importance of help from other players. Most of the interviewees (8) stated that they also received help from other players through global chat, team chat, the wiki, or from a specific person.

“The people on global chat were eager to help. Most of my experience I got from the help of my (former) team members.” (FD3)

However two players talked of teaching themselves, or preferring to figure out the game without any help from others.

“I tend to be a ‘lone wolf’, so to speak and I like to figure things out for myself.” (FD9)

159

Not only does a player learn the basics of the game tools, they also start to develop playing strategies and preferences. The player learns what types of puzzles are more appealing, where their strengths lie, and to what extent they want to work with others.

One interviewee described Foldit as a game where you have to find your own rules.

“Foldit players don’t have the same background and follow different paths and mental representations to find one good solution….you have to let people find their own way.”

(FD5)

One subject that was raised by several interviewees related to their attitude toward helping other players learn how to play. Two interviewees stated that they did not help new players due to a belief that a new player needs to find their own way in the game.

“I do not help others to learn. The really good ones will know how to learn themselves.

Helping makes them lazy, it is an everyday sight in the chatroom. Appreciation for help is not often expressed. I do not waste my time and energy on those who do not have the right stuff’.” (FD8)

Other interviewees expressed an interest in helping new players, and actively contributed to the wiki, or to online discussions.

“I’m more than willing to help others who ask for assistance (I think I’m a bit of a control freak, but don’t tell anyone).” (FD9)

An important part of learning to play for some players involves seeking out information relating to the science of protein folding, and most interviewees have increased their scientific knowledge through their involvement in the game. This information can be through informal channels, such as researching protein folding on the Internet, from other players, or from the information that is provided on the Foldit website. For one 160 interviewee, Foldit inspired him to return to formal education in order to understand the related science in more detail.

“I knew nothing about chemistry or microbiology or proteins or protein structures before joining Foldit. As a result of my participation I went back to school and audited two classes in chemistry and organic chemistry to get a better understanding of the forces at work.” (FD1)

The learning process is not restricted to those entering the game for the first time, and the game has changed since its launch in 2008. One of the most significant developments has been the increasing automation of some of the tasks. Instead of instigating puzzle moves ‘by hand’ one at a time, a sequence of moves can now be programmed using the scripting language Lua (see Section 4.1.1). These programmed sequences are known as recipes, and their application means the game can be speeded up and can be played even while the player is not physically present. Some of the players interviewed had strong opinions about this development, and it is evident that this change has had a significant impact on the game. Many players have learnt how to code in Lua and develop their own recipes, while others have remained ‘hand-folders’.

Ultimately, Foldit players use many different approaches in learning how to play. The game is not static and players must evolve with the game. The difficulty associated with learning how to play, and the complexity of the game, keeps the barrier to entry high, and ensures that only those with the ‘right stuff’ become long-term folders.

The Foldit Community

The success of Foldit can be directly attributed to its community of participants. Foldit players come together to work in teams, or through the global chat facility on the game interface. They share ideas and recipes while ultimately advancing the goals of the 161 project. All of the participants interviewed interact with the wider community of active players, and for many this is one of the most important and enjoyable aspects of their involvement. Even players who do not belong to a team interact with other players on global chat, or help new players. Some interviewees talked about the pride they have in the Foldit community and in its achievements, about the friendships they had formed and about the respect they had for their fellow team mates (one respondent referred to his team as his “folding family”). The ability to work both cooperatively and collaboratively with others is a core ingredient of the underlying player skill set.

“I like seeing how far we, my group and the whole community, can push the puzzles. I have made several friends in the Foldit community.” (FD6)

“Being in a group, team play and contributing to the benefit of the group is important to me.” (FD1)

One interviewee mentioned that the Foldit community is a good example of the ‘Pareto

Principle’ in operation. Also known as the ‘law of the vital few’, this phenomenon is observed when the bulk of the work in a community is carried out by a minority of its members.

“The Pareto Principle is alive and well on Foldit. A few folders make a huge difference to the rest of us.” (FD4)

The unequal contribution of project participants has been observed in all three projects that have been investigated as part of this research (see Section 4.1.3, Chapter Four), as well as in other online communities (Ciffolilli, 2003, Kittur et al., 2007, Preece and

Schneiderman 2009). It appears that some project participants have noticed this distinct pattern of participation also.

162

In addition to the interaction between players, there is some interaction with the scientists and developers from the project team. I asked the players how and if they interacted with the project team, and what they thought their wider role was within the project. Most respondents regard the management team as responsible for the overall co-ordination of the project, to supply participants with new puzzles, and to make sure the efforts of the players have some useful application. Two players stated that receiving feedback from the project team to the issues raised by the players is very important, yet one was slightly critical of the effectiveness of the response of the project team.

“I offered a few ideas and posted a little on the forum….however, little positive reply. I would have liked a pat on the back once in a while to keep me going.” (FD2)

Another interviewee expressed a desire to know more about how the contribution of players was advancing the science of protein folding.

“It is a bit frustrating that it is not very clear what the contribution to science is as a result of our efforts. “ (FD3)

Several players had little or no interaction with the management team, and did not really give them too much thought during their participation in the project.

“To be honest, I don’t give much thought to them. They are ‘over there’ so to speak. If they pop into chat to ask something, I will respond, but mainly I see them as the fixers of problems, or the givers of news. And the disher-outer of puzzles.” (FD9)

One of the respondents is a former number one ranked player, and he was the only interviewee in regular contact with the key project managers.

163

“I am in direct contact with Firas and Seth whenever I need contact with them or they with me. That should not change”. (FD8)

The ties between the players, particularly those on the same team, appear to be more developed and closer than those between the players and the management team. There also appears greater evidence of collaboration between players, than between the project team and the players. However, there is evidence of collaboration between some members of the core group and the project team, and this can be observed within the transcripts of the developer and scientists’ chats57.

Why I Play Foldit

While the Foldit community is an important general theme that has emerged from the results of the player interviews, it is also an important motivation to play. For seven interviewees, being part of a diverse community with a shared goal is one of the most enjoyable aspects of their participation. This motivator has also been found among

Galaxy Zoo participants (Raddick et al., 2010) and in participants in distributed computing projects (Holohan and Garg, 2005). Five interviewees stated that working with others towards a common goal, and getting to interact with participants from all over the world are key motivators for their individual participation in the project.

“Being in a group, team play and contributing to the benefit of the group is important to me.” (FD1)

“I have made several friends in the Foldit community.” (FD6)

A few of the interviewees expressed a desire to help, and had been personally affected by a disease caused by protein mis-folding (such as Alzheimer’s). Foldit may be able to shed

57 This transcript of a developers chat shows players and developers working on a problem relating to the in-game scoring system: http://fold.it/portal/node/991799. 164 more light on such diseases in the future, and many wanted to be actively involved in helping to find a cure.

“A family member fell ill with dementia and I wanted to help by searching for drug treatments. There were none and I stumbled upon Foldit.” (FD8)

“I was diagnosed with relapsing-remitting MS about 4 years ago. So I am interested in medical research, particularly oligo-dendrocites- the cells that re-build myelin.” (FD10).

Despite the more serious side of Foldit and its potential applications, several respondents stated that playing Foldit is a lot of fun, and a source of enjoyment.

“The game is great fun. And it is nice that it might help science.” (FD3)

Another important motivation is the opportunity to take part in authentic scientific research, and to make a contribution to a project with tangible outcomes. The fact that this could be achieved without a scientific qualification was very important to one of the interviewees.

“..the real point is that Foldit simply allows us folks without the proper CVs, and would crawl over broken glass to participate given half the chance, an opportunity to do this stuff. It’s that simple.” (FD4)

Some interviewees enjoy the intellectual challenge of the game, and enjoy the opportunity to develop their skills.

“I am also intellectually curious about the progress the Foldit approach to solving this problem has taken….I now have a cursory understanding of the forces at work and some theories about what might make a successful strategy.” (FD1)

165

Community and motivation are closely related within this small population of Foldit players, as is the desire to make an intellectual contribution to the project. Motivations are also in turn, influenced by the personal attributes, characteristics and interests of the players (such as perseverance, creativity, scientific inquisitiveness), linking this theme with the pre-requisite of the “right stuff”.

Discussions relating to why people play have also highlighted another important issue: how participants view their contribution to Foldit, and if they feel as though they are

‘doing’ scientific research. How a player views their contribution may have a direct impact on how long they stay committed to a project and the quality of their work.

Among the players interviewed, there are differing views on individual contribution, as well as differences of opinion regarding whether participating in Foldit constitutes carrying out scientific research. Most of the interviewees clearly felt that what they were contributing, or indeed, participating in scientific research.

“It does feel like I’m contributing to “SCIENCE”! Even if the details are a little nebulous.”

(FD9)

“I feel like I’m doing core uncredentialed science when I’m doing Foldit, which I strongly feel is a valuable adjunct to science done by trained scientists.” (FD4),

One interviewee described the role of Foldit players as similar to that of other ‘support staff’.

“We play a part. I don’t think we are doing science….it became scientific when it is analysed and prove to be true or useful by scientific methods. We help scientists to solve a problem like many other technicians useful in scientific research: the one who makes the

166 instruments or takes care of animals in a lab for example or the communication team…all these people are useful parts of the scientific work but are not scientists.” (FD5)

Several respondents were less sure about their contribution to science.

“For me it doesn’t feel like I’m doing science. It is a game and that’s also the idea the team want to promote.” (FD7)

“I’m not sure how much contribution I’m making as I’m not in the top 20.” (FD10)

Regardless of the degree to which interviewees felt they were ‘doing’ scientific research, some use language that is associated with scientific methodology and speak of developing and testing their “theories and assumptions”; “theorycrafting” with other players; and suggesting directions that the project may take. Many also confidently use technical terms associated with the science of protein biochemistry, and are able to hold productive and well-informed discussions with members of the Foldit team (some of the top ranking players in this small sample, work closely with members of the Foldit team).

For five of the interviewees, the opportunity to get involved in science is what draws them to Foldit, the degree to which they see themselves as participating in science varies from one player to another. It is unclear from the interview data why this may be the case. From a closer examination of the demographic characteristics of the interviewees, it does not appear to be related to player rank, level of involvement in the project or to their level of formal science education. Their perceived role in the project could be related to their understanding or interpretation of what it means to be a scientist and to conduct scientific research, however, this was not explored in any detail during this research.

167

Science as a Game

The games interface is a powerful draw for several players, and the system of points and ranking is a motivating factor for two of those interviewed.

“I liked the ranking system, it motivated me. Apart from that, the score on each puzzle also motivated me to try and get the highest score.” (FD2)

“I play by score; I have no real idea what I am doing, I just follow the score.” (FD6)

However, for one of the players, the competitive nature of the game may possibly interfere with the overall aims of the project.

“The players within the teams have collective knowledge in the true sense of the phrase, but their driving goal is, I feel, one of getting the highest score, rather than designing an actual working protein. (FD9)

Another respondent talked about the conflict between cooperation and competition in the game, and also made reference to the approach taken by ‘gamers’ – those he felt were more motivated by score instead of the scientific objective of creating a stable protein. One interviewee took exception to Foldit being called a game, and felt that it trivialised the efforts of the participants.

“Folding is work, hard work, NOT gameplay.

I am involved in scientific research. I approach it as such. If this really was just a game I would stop today and dedicate my intellect, time, money and determination to a more worthy cause.” (FD8)

168

However, this tension does not appear to be an issue for other interviewees and one respondent described Foldit as “the synthesis of crowdsourcing, scientific discovery and play.”

Packaging a scientific research problem as an online game may not be without its problems, and the potential ‘conflict’ between co-operation and competition may be an important consequence of this approach to online citizen science. This is encapsulated in the feedback of one of the interviewees who made the distinction between ‘gamers’

(who are motivated by score and rank) and ‘other’ types of participants (who are motivated by the scientific goals of the project). An examination of other citizen science games may shed further light on this issue.

5.2.2 Interviews with members of the Foldit team

Three interviews were carried out with members of the Foldit team58. All three of these individuals are software developers for the project (including one of the original developers) who work with both the Foldit playing community and with the biochemists at Professor David Baker’s lab at the University of Washington where outputs of the game are utilised. Since being interviewed, one of these individuals has since left the project and the other two developers I interviewed were happy to be paraphrased. Transcripts from these interviews was analysed in exactly the same way as all other interviews (Braun and Clarke, 2006). This was the outcome of numerous attempts over an eighteen month period to speak to the scientists who are involved in the project. Table 5.3 details the emerging themes from the interviews with the Foldit developers.

58 Interview questions are listed in Appendix F. 169

Table 5.3: Emerging themes from interviews with Foldit developers

The role of the project team The perceived roles and responsibilities of the developers in the Foldit project team. Retention and recruitment of players The issues surrounding retention and recruitment of players in Foldit as a consequence of its relatively high threshold for participation. Characteristics of the Foldit community The defining features of the Foldit community of players as viewed by the developers. How they work together, the development of smaller communities and specialisations. Community relations The issues that arise when working with a diverse group of volunteers. View of the player’s motivations Why do the project team think that people take part in Foldit? What are the perceived benefits for the volunteers?

The role of the project team

While the project developers are not scientists directly utilising the results, they have played a crucial role in the design, development, and implementation of the game

(Cooper et al., 2010, Cooper, 2011). The developers invest a great amount of time in liaising with the player community and in responding to technical issues that arise from time to time. From my observations of the player forum, and the transcripts of the developer’s chats, a player (particularly a member of the core group) is as likely to come into contact with one of the developers as they are with one of the scientists from the

Baker Group.

While the developers I interviewed are not scientists, two of them stated that one of the main roles of the project team was to help advance scientific research.

“The main role of the Foldit management team is to advance Foldit for the benefit of scientific research.” (FDPT1)

170

Liaising with scientists of the Baker lab is clearly an important aspect of the development team’s remit and two of the developers spoke of the need to work with them on a regular basis to discuss results, and what the players were doing. Scientists and developers also work together to explore new areas of research that could potentially be turned into a

Foldit puzzle. While the developers do not have a scientific background, they have inevitably learned a fair amount of biochemistry through their involvement with Foldit.

In addition to liaising with the scientists, the project team must also interact with the community. One developer stated that they work very hard to listen to all the feedback, and make changes where they can. Another developer stated that they must also put the needs of the scientists and of the research across to players.

“[project members] have done incredible amounts of work to help the community understand the needs of the scientists and how they are helping science.” (FDPT1)

However, this same individual went on to state that they occasionally use too much scientific or technical jargon, and that they may need to work more to make the science behind the project more accessible to players.

“I think we can do a better job at explaining what it is exactly people are doing at any given time.” (FDPT1)

In addition to working with the project scientists, the developers also need to ensure that

Foldit is fun and enjoyable for those who are committed players and that it can attract new players (Yee, 2006, 2007).

Recruitment and retention

One of the developers is actively involved in communicating and promoting Foldit. There have been numerous news articles written about Foldit since it was launched in 2008, as 171 well as features on radio and television programmes. The appearance of these items, particularly on radio and television often result in an increase in the number of registered players and people taking a look at the game. I observed increased activity on the game and in global chat after Foldit was featured on a national US radio feature, and on a US television documentary.

However, as there is a lengthy learning process associated with Foldit, most of this interest does not always manifest itself as a significant increase in the playing community.

According to two of the developers interviewed, most people are ‘lost’ in the tutorial puzzles, and each successive puzzle has fewer players than the last. The developers are able to observe where, and at what stage there are the biggest number of losses, and work to try and address this. Foldit is a complex game, and the tutorial puzzles teach potential players about the project tools, so there aren’t always changes that can be easily made.

There appears to be a general acceptance among the developers that Foldit will not appeal to all who take a look at it. One developer spoke of focussing on the current core community, and keeping the game interesting for these individuals.

Characteristics of the Foldit community

The developers stressed that the community of players is crucial to the success of the project. One of the advantages of opening this research up to the wider community is that it fosters diversity in the approach to the protein puzzles. Participants bring different skills and problem solving experience to the game which has been beneficial to the scientists at the Baker Lab. One of the developers stated that it is this diversity that has helped the game to advance scientific research.

172

“I think the main benefit of working with ‘citizen scientists’ is the diverse set of knowledge and skills you have at your disposal.” (FDPT1)

There is also an acknowledgement that the skills of the Foldit community have surpassed the developers when it actually comes to playing the game. They have developed skills, new tools, and strategies over the past five years (for example, the development and coding of ‘recipes’, the automated sequences of moves), and the project has evolved as a result. While the remit from the scientists from the Baker Lab may determine the overall research goals of the project, some members of the community have a degree of influence in determining how those goals are met by suggesting particular approaches via the developers’ chats, and by highlighting problems with the project software.

The developers referred to the small group of core players, and there was some agreement with my estimate of the number of these individuals (20).

“There is a small group of players who contribute many hours to Foldit and that group tends to stay fairly static.” (FDPT1)

This group in particular has accumulated many skills over the years, and have been responsible for one of the more notable changes in the game, the development of recipes. The impetus for this development came entirely from the players, and the developers assisted them in developing this new way of playing. This has changed the nature of the game for many, and according to one of the developers has added a new layer of competition to the game, as individuals work to create the ‘best’ recipes.

The discussion about recipes led to a consideration of the competitive element of the game and how this interplays with the desire to co-operate with other players and work towards the ultimate goals of the project. While players strive to design the best recipes,

173 they will also share these recipes with members of their own teams, or with the rest of the Foldit community. The same can be said of the solutions to the puzzles which can also be shared. One of the defining features of the Foldit community that emerged from these developer discussions was that competition and co-operation sit side by side.

Not all Foldit players use recipes when they are playing, and some remain ‘hand-folders’ and carry out all of the necessary moves and tweaks manually using the set of tools provided by the developers. One developer referred to different “ideologies” that have emerged within the community, as some players prefer and adhere to different strategies of playing. The developers try to accommodate these different playing approaches, and will sometimes set puzzles where recipes can’t be used, and have to be completed by hand-folding alone.

The Foldit community has organised itself into smaller groups and teams. Not all players belong to teams, but some of the ‘solo’ players within the core group still interact and work with other players in the global chat window. The developers noted that there are always opportunities to interact with other players, and to help others along the way.

Community relations

There can be substantial interaction between the team of developers and the Foldit community particularly with the core players (e.g. through developers chats, and the

‘Feedback’ area of the website), and at times, this is one of the more challenging aspects of being involved in the management of the project for the developers. However, maintaining this level of interaction has led to significant developments and improvements in the game.

174

While having a diverse approach to problem solving is beneficial to the project in terms of problem solving, in can also lead to problems.

“That same diverse set of people has a very diverse set of opinions on what is the ‘correct’ path when it comes to issues that arise……what we think may be best for the community may not be what some of the community thinks is best for the community.” (FDPT1)

This developer spoke of spending time dealing with community relations, and of his attempts at dealing with the community in a sensitive manner, and the fact that some of the players could be “easily offended”. Another developer stated that some of the interaction between the developers and players could become quite difficult at times and that one had to not take feedback personally and should consider what issues were actually being raised in such exchanges. Two of the developers spoke about not being able to please all of the players all of the time.

The importance of interacting with the community, rather than controlling or managing them, was stressed by one of the developers.

“I don’t think the players need to be actively managed….I think active involvement in the community helps immensely.” (FDPT1

The same developer spoke about the appointment of a community relations manager in

Foldit, and how that was helping to “give them a voice” and enabling them to get more involved in what the developers do on a day-to-day basis.

View of the player’s motivations

Why people become involved in Foldit and commit significant amounts of time are clearly of interest to the project management team. All of the developers had clear views as to why they thought people got involved. 175

“One of the main driving forces is the altruism of players, many want to advance biochemistry and see Foldit as way to do so. Some players have loved ones who are afflicted by diseases that have or may be addressed by Foldit.” (FDPT1)

The desire to contribute to science was also considered to be a main motivating factor by the other two developers. Being able to contribute was thought to provide the players with a sense of fulfilment. Foldit was also described as a “game with a purpose”, and players could play knowing that they aren’t just wasting their time, but are actually making meaningful contributions to scientific research. Two of the developers spoke of the fact that players hear about Foldit through “science-based” channels such as popular science magazine articles, in the science or technology sections of newspapers, or science-based websites, so it naturally attracts those that are interested in science.

The competitive, or gaming, aspect of Foldit was felt by one of the developers to be one of the most important draws for participants. The competitive element also meant that even if you weren’t doing that well as an individual player, your team could be doing well in the team rankings. However, another developer stated that he didn’t think this aspect of the game was important to everyone.

“The competitive aspect of Foldit is also a major selling point. I don’t think it is the primary factor for many players, but it is a hook that keeps people more involved in the community.” (FDPT1)

Many players remain with Foldit because of the community, and because they make friendships with other players.

176

“As Foldit is a large community, many subgroups have formed. These groups can be more personal and intimate, fostering another reason to come back, to meet and talk with your new found friends in the group.” (FDPT1)

These groups also make it possible for players to help others and co-operate while working on the protein puzzles. This is also an enjoyable aspect of the game – perhaps one that appeals more to those who aren’t so motivated by the competitive aspect according one of the developers.

One of the developers spoke about the importance of being aware of all the different motivations for participation in Foldit, and how these factors needed to be taken into consideration during the design of the game.

Despite the fact that these developers do not directly use the results generated by Foldit players, they also have motivations behind their participation.

One developer wanted to work with those who were external to academia.

“I wanted to work on a project that was not only being used for research purposes, but also had a wider reach among the non-research community.” (FDPT1)

One of the other developers who had had a major role in the development of Foldit, was more enthusiastic about the use of games in scientific research, and was clearly keen to explore this in his work with Foldit. He spoke about how public perception of computer games had changed in recent years, and that there was a greater appreciation of the other areas (apart from entertainment) when games could be useful, such as carrying out scientific research.

177

5.3 Summary of results

The feedback from the survey illustrates that this group of respondents is predominantly male, aged over 40, well-educated, and with a background interest in science. These findings have revealed more about the characteristics of some of the active playing group and help to address RQ1 (Who participates in online citizen science?). Most of those surveyed are regular, committed participants who spend at least fifteen hours a week playing Foldit. Within this sample there are some members of the core group of players, who serve as ‘leaders’ in this community (Preece and Schneiderman, 2009). Thus, the survey feedback has provided a quantitative measure of participation and has helped to address RQ5 (How can contribution to online citizen science projects be characterised?).

The survey has also provided information on how participants interact online (RQ4). The majority of respondents play in teams, and regularly use internet relay chat to communicate with, and work with other players.

Respondents are motivated to begin playing because they want to make a contribution to science, and have an interest in science. They also like to develop new skills and some like the competitive aspect of the game. Making a contribution to science keeps individuals playing Foldit, as does the enjoyment they derive from interacting and being part of a community. These findings have helped to address RQ2 (What motivations initiate and sustain participation in online citizen science projects?), and have demonstrated some similarities with the findings of other studies (Holohan and Garg,

2005, Raddick et al., 2010, Krebs, 2010). As in the research by Rotman et al. (2012), motivations of players are dynamic and can change over time, and the importance of the

Foldit community emerges as an important motivator for sustained participation.

178

The interviews with players have also highlighted the importance of having the ‘right’ combination of skills and personality traits. For example pattern recognition skills, intellectual curiosity, and an interest in science, combined with patience and perseverance. This suggests that those with these skills are more likely to be among the group of active and core players (RQ1). Feedback from the player interviews also highlighted the importance of the community, and how help from other players can be of importance during the process of learning how to play. The latter observation has been observed in previous work on how participants in Zooniverse projects learn about the project tasks (Mugar et al., 2014). More detailed information regarding the interaction between players, and between players and the project team has been highlighted (RQ4) and interview feedback confirms, as noted in my observations (Chapter Four, Section

4.1.5), that Foldit players exhibit a degree of independence from the project team. How participants perceive their role in the game (RQ6) has been explored. While a few players consider themselves to be actively participating in scientific research, most felt that they were providing more of a supporting role to the project scientists.

Feedback from the developer’s interviews suggests that there is a core group of players who make a significant contribution to the project. These individuals are extremely skilled and their abilities in the game have overtaken those of the project team. The game is very difficult to learn and most individuals who take a look at Foldit do not progress beyond the first few tutorial puzzles. Given the high threshold to participation, recruitment and retention of players has to be addressed on a continual basis. These factors are relevant with regard to motivating and sustaining participation. Relations with the Foldit community can sometimes be challenging, but the need for regular communication and engagement was recognised. This feedback provided some insight

179 into how players and members of the project team interact (RQ4), and illustrates that

Foldit developers aim to take account of, and to incorporate player feedback. The developers saw a real potential of computer games as a basis for citizen science projects which motivated their involvement in Foldit.

This data, along with my own experience as a player demonstrates that this is a complex and challenging game that appeals to individuals with specific skills and interests. A small group of active players invests a great deal of time in the game, are committed to the scientific objectives, and has developed their knowledge of protein biochemistry. The project community is tight-knit, with friendships forming among the players as they co- operate and collaborate on the science puzzles. Members of the project team rely on and value the input of players, not just to achieve the scientific objectives of the project, but also to improve and develop the game. Some players are very influential in this respect and work closely in partnership with the Foldit developers.

180

Chapter 6: Folding@home results This chapter presents the findings from the Folding player’s survey (see Appendix C) and a thematic analysis of the interviews with 15 participants and two members of the project team. A detailed analysis and a comparison with the other two projects will be presented in Chapter Eight.

6.1 The online survey results

This survey had the greatest number of respondents (407) which may be due to the fact that this project has the largest community of active participants out of all three projects.

The higher response may also be due to the fact that the link to my survey was independently shared on a number of Folding@home team web sites and other discussion forums (e.g. overclocker forums) by some of the survey respondents.

However, as outlined in Chapter Four (Section 4.2.4), the actual number of participants in

Folding@home is difficult to ascertain, and various estimates range from 27 000 to

100 000. Therefore, this sample is still a relatively small proportion of the total number of participants, and represents approximately 1.5% of participant numbers (if the lower estimate is taken). However, it may represent of a greater percentage (4%) of the more active community of overclockers and hardware enthusiasts, which has been estimated to number 10 000.

The results of the survey have been broken down into four main areas: demographic characteristics; patterns of participation; motivation and reward; interaction with other players; and views regarding contribution. For the results of the each individual survey question presented, n=407 unless otherwise stated.

181

6.1.1 Demographic characteristics

The majority of respondents (63%, 255 individuals) were under the age of 40 (unlike

Foldit). Most were based in the US, Canada and Europe. Only 11 respondents were from developing countries. However, one of the most striking results from this survey was the very small proportion of female respondents. Less than 2% were women (seven individuals in total). Of the studies that have looked at distributed computing projects, all have found that the majority of respondents to their surveys (usually over 90%) have been male (Anderson, 2004, Estrada et al., 2013, Krebs, 2010, World Community Grid,

2013).

The disproportionate representation of men may be due to the appeal of distributed computing projects to hardware enthusiasts, and those with an interest (professional or amateur) in computing. While significant efforts have been made to improve the representation of women in IT-related industries, and in their uptake of computer science at school and university, women continue to be under-represented in this field

(Commission, 2012, Camp, 2012, Klawe et al., 2009). This may have a knock-on effect regarding the numbers of women who are computer hardware enthusiasts and who take part in distributed computing projects, or even have an awareness of such projects. Of the seven women who responded to my survey, two are currently studying IT and electrical engineering, and two are IT professionals who describe their computing skills as advanced.

Just over half of respondents (232) had a university degree, while approximately 26%

(106) were educated up to high school or UK ‘A’ level (see Figure 6.1). However, 80 respondents (almost 20%) are currently students, which may explain the younger age

182 profile of this group of participants. Of those who had a university degree, the vast majority (82%) qualified in a STEM subject.

Figure 6.1 Highest educational level attained by respondents

high school (UK GCSE) 23% 19% UK 'A' levels / BTEC junior college (HND) 3% some college course undergraduate degree 14% postgraduate degree 35% other 2%

The majority of respondents are in skilled professions. A significant proportion (37%, 150 individuals) stated that they work in an IT-related profession (see Table 6.1). A high representation of IT professionals was also observed in the World Community Grid survey

(2013). There is also a significant representation (about 12.5%) from science or engineering-related professions.

Table 6.1 Occupation / profession of respondents (n=403)

IT Professional 150 Student 80 Business professional 43 Engineer (not IT) 26 Science / medical 24 Technical / mechanical 19 Unemployed (no previous occupation given) 16 Clerical / admin 8 Retired (no previous occupation given) 6 Retail 5 Education 5 Law enforcement / fire service 4 Military 3 other 14 183

6.1.2 Background interest in science and participation in related activities

Over half of respondents (212) had not participated in any other citizen science projects, and Folding@home was their first experience (see Figure 6.2). Nearly half (179) had taken part in other distributed computing projects with many respondents mentioning

SETI@home and other BOINC projects such as the World Community Grid and

Einstein@home. Participation in other types of citizen science project was limited with a small number (12) trying Zooniverse projects and Foldit.

Figure 6.2 Participation in other citizen science projects

212

179

6 6 2 2 1

* distributed computing

Most respondents had taken part in a variety of science related activities in the previous year (Figure 6.3). Watching scientific television programmes and reading online science material were the most popular. Many read science magazines, and nearly a quarter had taken part in an amateur astronomy event during the past year.

184

Figure 6.3 Other science-related activities undertaken during the last 12 months

290 269

160 153 122 106 104

30 24

6.1.3 Patterns of participation

Sixty per cent of respondents (244) stated that they had been taking part in

Folding@home for over 2 years (see Figure 6.4). Sixty-one respondents (15%) were relatively new and had been participating for less than 6 months. I included an option for members of the Beta Testers, and members of the Professor Pande’s research group who were also participants. This was in order to help identify those individuals who may be making a greater contribution to the project. In total, 32 respondents are on the Beta

Testers, and five are members of the Pande Group.

185

Figure 6.4 Duration of participation in Folding@home

Pande Group member 1% Beta Tester < 6 months 8% 15% 6 months - 1 year 7%

1-2 years 9% > 2 years 60%

The survey included questions about participants’ involvement in Folding@home online discussions and forums. From reviewing the responses, it became apparent that interaction between participants is not just restricted to the ‘official’ Folding@home forum, but also takes place on the forums of some of the larger teams, and among other forums associated with overclocking and hardware enthusiasts (such as supplier and manufacturer’s forums). Sixty-five per cent of respondents (264 individuals) report using the ‘official’ Folding@home forum, while 10% of participants (40) report the use other forums (such as overclocking and enthusiasts’ forums).

Five per cent of respondents (20) specified that they used both the official forum, and other discussion forums (see Figure 6.5). However, this figure may in fact be higher given the wording of the original question (“Do you participate in the Folding@home online discussions and forum…”), as some may have thought I was asking specifically about the official folding@home survey. Only 20% of (81) respondents stated that they didn’t participate, but again, this may be inaccurate as some of these respondents may actually participate in other forums and not the ‘official’ one. A closer inspection of the ‘no’ 186 group reveals that 67 of the 81, are quite specific in their response stating that they were unaware of the forums, didn’t have the time or expertise to contribute, or simply wanted to run the software and not get any more involved in the project.

Figure 6.5 Participation in online in Folding@Home discussions and forums (n=405)

do not participate in 5% forums 20% use other forums to discuss F@H (not offcial F@H forum) 10% use 'official' forum

65% use numerous forums to discuss F@H

Respondents who participate in online discussions can spend a significant amount of time interacting with other project participants. Nearly 20% spend over 5 hours a week on forums. A closer look at this group suggests that some of this interaction occurs on the

Folding@home discussion threads on overclocking websites (for example [H]ardOCP59,

EVGA60, OverclockersUK61). These forums provide a space for enthusiasts to help each other build better performing machines by sharing advice and expertise, and thus help participants increase the amount of points they accumulate on the project. However,

42% of respondents (170) who participate in forums do so for less than an hour a week.

Most of the survey respondents (89%, 362 respondents) belong to a team and some respondents mention participating in team forums. Many of these respondents

59 [H]ardOCP Folding@home forum : http://hardforum.com/forumdisplay.php?f=32 60 EVGA Folding@home forum : http://forums.evga.com/tt.aspx?forumid=28 61 Overclockers UK DC forum : http://forums.overclockers.co.uk/forumdisplay.php?f=39 187

(approximately 250) appear to belong to a team that is obviously related to an overclocking, hardware enthusiast, or gaming community. For example: Team EVGA62;

Maximum PC; and Team OCF (over clockers forum) to name a few. Some belong to national teams of hardware enthusiasts such as Dutch Power Cows and Hardware.no

(Norway).

6.1.4 Motivation and reward

Respondents were asked why they decided to participate in Folding@home. This was to address the first part of RQ1 concerning motivations that initiate participation in an online citizen science project. The responses to this question are illustrated in Figure 6.6.

Figure 6.6 Why respondents decided to participate in Folding@home

207

128

74 68

27 21 16 12 11 2

62 EVGA is a hardware and motherboard manufacturer 188

Folding@home respondents usually had more than one reason for taking part in the project although the most commonly cited reason for participation (49%, 207 individuals) is to make a contribution of some sort. There appeared to be two distinct types of contribution: contributing to a ‘worthy’ cause; and making a contribution to scientific or medical research. Over a quarter of respondents (118) state that making a contribution to a worthy cause is one of the reasons they began participating in Folding@home. This is a distinct motivation to contributing to scientific or medical research, as many respondents view their participation as a type of a charitable donation – only they are donating their computer processing power instead of cash. Other respondents (89) refer specifically to making a contribution to science, and like to feel as though they are part of a wider research endeavour.

The second most commonly cited reason for participation (31%, 128 respondents) was to fully utilise computing power, and this is a reflection of the presence of overclockers and hardware enthusiasts among this group of participants. Fifty respondents specifically mentioned their involvement with an enthusiast community in their response to this question, while many in this group made comments regarding minimal wastage of power or getting the most out of their machines.

Another important reason that respondents participate in Folding@home is because they have had a personal experience of one of the diseases that is being researched through the project, either personally, or with a family member. Seventy-four respondents (18%) relate some experience of cancer, Alzheimer’s Disease, Parkinson’s Disease, or other degenerative disease, and through their participation in Folding@home, they address a need to take a more ‘active’ role in potentially beneficial research.

189

Few respondents state that they join the project for the community or for the competition, which is surprising given the involvement of overclocking community.

However, in a previous study of distributed computing volunteers, the authors noted that respondents had ‘official’ reasons for taking part, which were often altruistic, and

‘unofficial’ reasons, which were often implied and usually related to the competitive aspect of participation and position on leader boards (Holohan and Garg, 2005). This observation may also be of relevance to Folding@home, and is also alluded to in the interview feedback (see Section 6.2.1).

The next question was what they liked best about Folding@home. The responses to this question are illustrated in Figure 6.7.

Figure 6.7 What respondents like best about Folding@Home

150

75 54 47 39 37 26 22 18 15 8

Making a contribution either to scientific or medical research or to the project as a

‘worthy cause’ for approximately 150 (or 37%) of respondents, and the ease of using

190

Folding@home is appealing to many respondents, both within and outside the hardware community. Respondents who are more technically inclined talked about specific technical features of the project that they liked.

Community and competition were mentioned more in relation to sustaining participation than in relation to initiating it, and approximately 10% of the respondents mentioned that these aspects were what they liked best about the project. Some of the communities referred to were those within the population of hardware enthusiasts or to members of the same Folding@home team.

Respondents were asked whether they thought Folding@home participants should be rewarded for taking part, and if so, what would be the most appropriate way of rewarding them. More than half of the respondents (248) feel that there is no need to reward participants beyond what is already offered (e.g. points for completed work units and the chance to participate in a scientific research project), and that participation is its own reward.

While a majority felt no extra reward was required for participation, over a hundred participants (25%) would like to see something extra offered to participants and made suggestions as to what that should be. These included virtual badges, better quality certificates (currently participants can download a certificate of participation at any time), tours of the labs for significant contributors, an annual convention for participants, discounts on related computing products, discounts on educational items, cash for reaching a certain level of points, prizes such as Folding@home T-shirts or mugs, or a

‘user’ of the day feature (this is done on many BOINC projects). However, the most commonly suggested reward (made by 31 respondents) was to be able to claim their

191 electricity costs as a tax rebate, in the same way that other charitable donations can be offset (this suggestion was specific to US participants).

In addition to tax rebates, approximately 20 respondents indicated that they would like to see a greater acknowledgement of the contribution of participants in scientific papers. A similar number of respondents stated they would like to receive more information about the project.

6.1.5 Interaction with other participants

Compared with other types of online citizen science projects, the opportunities for interaction between participants in a distributed computing project may appear at first to be limited. As a small number of respondents to the survey have pointed out that for them, participation simply involves downloading the software and forgetting about it. A few respondents were even unaware of the existence of the Folding@home forum.

It is within the hardware enthusiast and overclocking communities that many participants are able to make significant contributions to the project. Their expertise (and willingness to shoulder higher electricity bills63) has meant that the output of the project is significantly greater than it would be without their involvement. Where this extra contribution occurs, there are also opportunities for cooperation or collaboration between these participants as they strive to improve the performance of their machines.

Survey respondents were asked: Do you have any interaction with other participants within the project, or with members of the Folding@home scientific team? If yes, how would you describe that interaction?

63 The amount of electricity consumed and subsequent costs will vary according to components and how many hours per day a computer is running Folding@home. This topic is commonly discussed on overclocking forums, as are ways to keep costs as low as possible. 192

A significant proportion of respondents (41.5%, 169 individuals) report little or no interaction with other participants. By ‘little’ this means occasionally reading the posts on the Folding@home forum, or other parts of the project website. Over 58% of respondents report some interaction with other project participants. However, not all of them describe this interaction (or specify where it takes place), but those that do, mainly report positive and helpful interchanges with other participants. Much of the reported interaction occurs within the context of a team forum, or on discussion forums that are frequented by hardware enthusiasts or overclockers. Over 100 (25%) respondents report involvement in such discussions.

In addition to the team and overclocking forums, there is the ‘official’ Folding@home forum. Some respondents interact on this forum in addition to team and technical forums, while for some respondents, this is their only online venue for Folding@home- related discussions.

Very few respondents report any direct contact with members of the project team.

Those that do are usually members of the Beta Testers as they work together to make improvements to the project software. The Beta Testers will be explored further in

Section 6.2.1.

6.1.6 Perceived contribution to Folding@home

Question 12 on the survey asked whether participants felt as though they were involved in scientific research. In total I received 366 responses that went some way to addressing the question, although some of these were just ‘yes’ or ‘no’ answers. The responses have been classified into six broader categories (see Figure 6.8).

193

Figure 6.8 Do respondents feel as though they are involved in scientific research

138

57 45 28 12

I feel involved I am only a I am I am just I don't feel I don't know in scientific small part of a supporting & donating involved in research wider effort enabling the resources scientific work of research scientists

Over a third of respondents (35%, 142 individuals) do indeed feel as though they are involved in scientific research – particularly if they have been participating in the project for a number of years, or if they are a high-ranking participant. While some respondents feel unequivocal about their involvement in scientific research, approximately 22% of respondents (90) describe their involvement as very small, and view it within the wider context of a project in which many thousands of individuals are contributing. Twenty- eight respondents (7%) describe their role in Folding@home as a supporting one, and make a clear distinction between actually doing the research and enabling it or facilitating it.

Fifty-seven respondents (14%) make the distinction between actually being involved in scientific research, and just making a contribution or donation. This group of respondents feel that they are merely donating their computing resources, or liken their involvement to making a cash donation to a charity. Approximately 11% (45) of respondents felt they

194 were not involved in scientific research. Many of these respondents did not connect what they were doing with their computers with the process of doing scientific research.

6.1.7 Further quantitative analysis

As outlined in Chapter Three (Section 3.4.1) a quantitative analysis of the survey results was attempted. The larger number of Folding@home respondents meant that some statistical analysis could be attempted. However, given the relative homogeneity of this group (particularly in relation to age and gender), only a limited amount of tests were undertaken. Chi-squared tests were carried out in order to explore possible relationships between participation in online forums (which was coded according to how many hours a week respondents participated) with the following: level of education; level of STEM education; time with the project; and profession (whether a participant was in IT or not).

Only one of these tests produced any significant results. The amount of time a respondent spends on the online forums is related to how long they have been with the project. The longer they have been with the project, the longer they spend on online forums (Chi-Square = 45.85, P<0.001).

6.2 Interview feedback

The results of the interviews have been subjected to a thematic analysis using the approach of Braun and Clarke (2006) which was covered in detail in Chapter Three

(Section 3.4.2). The major themes will be presented for both the interviews with citizen scientists (n=18) and with members of the project team (n=2). The emerging themes will be outlined64.

64 The interview questions can be found in Appendix E. 195

6.2.1 Interviews with citizen scientists

The online survey had identified the presence of two important sub-communities of participants in Folding@home: overclockers / hardware enthusiasts; and the Beta Testers.

Members of these two groups comprise the active participants in Folding@home, and I wanted to find out more about them. Overclockers in particular are important contributors to distributed computing projects, yet very little has been written about them (Bohannon, 2005). Therefore, subsequent interviews of citizen scientists focussed on members of these two groups.

During the analysis of the interview data, five themes were identified that were of relevance to all of the respondents, In addition, several themes were of importance to either the Beta Tester interviewees, or the overclocker interviewees. Table 6.2 outlines the themes from all 15 of the interview subjects.

Table 6.2 Emerging themes from interviews with Folding@home participants

What have I learned? What has been learned through participating in Folding@home, particularly with regard to the science of protein folding, and learning about the technical aspects of running a distributed computing project. Contributing to research Participants’ view of the way science is enabled by Folding@home. Openness and transparency, as well as opportunities to make a contribution to research. What I like about participating The more enjoyable aspects of participation and what keeps them participating in the project. The ‘mission’ Participants united in a community-wide effort to combat serious diseases. Personal experiences with the diseases that Folding@home investigates. Relationship with the project Issues relating to the way the project team communicates with the team community.

196

What have I learned?

All interviewees were asked if they had learned anything though their participation in

Folding@home. Approximately half stated that they had learned more about the science of protein folding either by seeking out and reading information, or more passively acquiring information after years of involvement in the project.

“I have learned about protein folding and how studying it can help with research for cancer and other serious diseases.” (FH13)

“Basic principles of protein folding structures. I admit most of the stuff is way over my head, but when you’re involved in something for so many years you automatically pick one or two things up.” (FH4)

Several respondents outlined how they had learned more about distributed computing projects, and the potential these have to address real-world scientific problems.

“Folding@home has taught me several lessons about how to use common technology for the greater good.” (FH8)

“The project has put an interest of distributed computing in me; I am currently enrolled in

‘introduction to Parallel Computing’ on Udacity to learn more.” (FH10)

Most of the respondents talk about what they have learned in relation to the technology

(hardware expertise or software skills) associated with their involvement. All of these individuals are technically proficient, and are involved in computing either professionally or through their hobbies. For some of these participants, learning about computing is one of the reasons they became involved in Folding@home in the first place, so the opportunity to learn more about hardware, the use of GPUs in non-gaming contexts and programming in Linux (necessary for some aspects of overclocking) is important to them. 197

“It introduced me to Linux earlier than I would have found it….I’ve learned more about the various components of a computer.” (FH11)

“One major aspect was my introduction to Linux. I never used it before and as I learnt about F@H, I realised Linux was quite useful for some aspects of F@H and eventually I learnt the basics of it….” (FH7)

Three individuals mention other things that they have learned that are related to working within a larger community towards a common cause such as developing team-building skills, and being able to work with others.

Contributing to research

Most of those interviewed are motivated to participate in Folding@home to make a contribution to science.

“As a child, I would dream of being a scientist but as I grew up, that dream just remained a dream. Thus, when the opportunity presented itself to me to help scientists find a potential cure to cancer, Alzheimer’s and other diseases, I was overjoyed and spent a considerable amount of time participating in the F@H project.” (FH7)

“Having a distributed network to do the ‘dirty work’ for them is immensely helpful. It allows them to do more science ‘for free’ and also focus on analysing the data. We facilitate the science.” (FH9)

Other views related to the science carried out by the Pande Group were also highlighted and the value the interviewees placed on the quality of the science became apparent. For example, several respondents felt that the work the Pande Group do is at the cutting- edge of science.

198

“Protein folding is at the cutting edge of science. I believe it will lead to many new drugs for diseases and possibly some cures.” (FH3)

Several other interviewees highlighted the fact that the results of the research were open, and available for any other research group to use.

“As the papers from folding are openly published for all researchers in the field to read, who knows how many accomplishments by other groups are inspired or aided by the result of folding at home.” (FH6)

From comments like these, it is not only the opportunity to contribute to science that is important to some of these participants, but also the opportunity to contribute to a project that is open to other researchers and where the results are regularly published.

This opportunity to contribute has been highlighted in previous studies of online citizen science (Holohan and Garg, 2005, Krebs, 2010, Raddick et al., 2010).

What I like about participating

For many of those interviewed (particularly those who describe themselves as overclockers), Folding@home gives them the opportunity to push their hardware to the limits, while also achieving something that is worthwhile.

“All of the money, electricity and my time is going toward a good cause while at the same time I get to explore new hardware and configurations.” (FH6)

“Rather than waste my time ‘online’ I have found a fun and educational way to spend my free time which has a nice balance of work and play.” (FH7)

Learning about new technology and the application of hardware is enjoyable for several interviewees.

199

“You learn about how new technology is helping scientists reach goals which were once deemed impossible.” (FH7)

Another respondent liked the fact that there is some degree of transparency regarding the results of their efforts.

“I have never been a fan of senselessly throwing money at a cause because I never know where the money is going. For all I know, the money I donate goes to buying packs of paper. By donating my processing power, I know exactly what my contribution is doing.”

(FH9)

Most respondents enjoy being involved in a larger community working towards a common goal.

“I fold to be part of a greater community in the Folding@home world – in this case, lately, as a member of the EVGA folding team where I have developed several years worth of friendships.” (FH 8).

“The people on this team are great folks – always supportive and willing to help solve issues regardless of what the problem is. If they don’t know the answer, they’ll help dig and find it. Along with the teammates, just the general folding community.” (FH9)

“I do enjoy interacting with others that are interested in this area.” (FH10)

This enjoyment can stem from their involvement with the Beta Testers, their own team, or just the general Folding@home community.

The ‘mission’

One of the respondents referred to the fact that he was involved in Folding@home because he had a “mission” and a “purpose”.

200

“Until everything we are fighting is gone, I will continue to fold.” (FH9)

This is a reference to the diseases that the Pande Group seek to understand as part of their research. In the online survey, approximately a fifth of respondents stated that they had been personally affected by the illnesses that Folding@home investigates. This personal connection to these illnesses was also referred to by the majority of the interviewees emphasising that for some participants, the long-term goals of the project are of great importance.

“Some of my closest relatives have been affected by some of the conditions researched by

Folding@home. I wish to contribute in whatever way I can to improve our medical knowledge of these conditions…” (FH12)

“I have experienced first-hand the effects of diseases like cancer and Alzheimer’s. I have lost family members to these diseases and have a good friend that has cancer, who I met through the folding community.” (FH13)

“My grandparents were affected with diseases that PG [Pande group] research, particularly Parkinson’s and Alzheimer’s, and I would sincerely hope that the PG’s efforts will lead to a cure.” (FH5)

“One day when / if folding@home data helps to eradicate few deadly diseases, I will proudly be able to tell everyone, that I helped finding a cure for this or that disease.” (FH4)

Even for those who do not mention a personal experience of the Folding@home diseases, the long-term research goals of the project are recognised as being important.

201

Relationship with the project team

Five of the interviewees who belong to the Beta Testers report some contact with members of the project team as they work to test new project software and identify any bugs. While most of those interviewed from the overclocking / enthusiasts’ community do not report any direct contact with the project team, some take an interest in project developments and in any change of policies or project parameters. As some overclockers invest significant amounts of time, energy and money (in the form of hardware and electricity) in Folding@home they like to maintain a level of involvement and keep up to date with the project team through the blog, and numerous folding forums.

Practically all of those interviewed had some opinion about the project team, and particularly how the team interacted with the folding@home community. Several respondents wanted to see better general communication with the folding community.

“The biggest thing is increased communication. What I really want to know is what is currently useful to the scientists and where they think the project is heading. It all comes back to knowing that the work we are doing is useful.” (FH6)

“More behind the scenes content. While donors only get to see the final product, it would be nice to see what happens within the labs.” (FH7)

The importance of communication in an organisational setting has been explored in the literature (Tourish and Robson, 2006, Salem 2008), and work on an online community that produces film reviews has also shown that communication should focus on the value of the contribution of community members (Rashid et al, 2006). One individual also stated that the scientists could be terse at times and sound “authoritarian”, and that this was most likely the result of them not having enough time to keep up with the folding

202 community on a regular basis. A feeling of an “us” (the folders) and “them” (the scientists) was also articulated by another individual.

“They have a history of not being too understanding with the concerns of the contributors.

This slightly elitist attitude is kind of a turn off. With folding it feels more like there are people in charge (like your boss at work) who hear what you say but don’t always seem to be listening.” (FH13)

A few of the comments about the project team were quite negative. However, it is very likely that some interview feedback was influenced by a change in the BigAdv (Big

Advanced Folding) points initiative (see Chapter Four, Section 4.2.6). These changes were announced in December 2013, and I carried out these interviews during the first few weeks of January 2014. A few comments directly referred to the BigAdv changes, and these respondents generally felt that changes were made without any consultation with the community, and felt undervalued as a result.

“…whenever change occurs in the project, how the change affects users are not really the primary focus. I would argue that too many people end up feeling badly about their donation. They feel that they are not valued and end up quitting and bad mouthing the whole project.” (FH2)

How points are awarded has become a contentious issue for several of these participants and is an important consideration for Folding@home as well as other distributed computing projects that rely on the efforts of overclockers. The reaction to BigAdv also suggests that some participants are highly motivated by points and by measuring the performance of their machines, something that wasn’t so obvious in the online survey results. Holohan and Garg (2005) in their work on distributed computing projects

203 identified ‘official’ reasons for taking part which were more altruistic, and ‘unofficial’ reasons which were based on pointes (see Chapter Two, Section 2.7.5). It may be that I am seeing a similar emergence of ‘official’ reasons for participating (such as making a contribution) and ‘unofficial’ reasons, which are more influenced by extrinsic (points) reward and competition with others.

The Folding@home Beta Testers

The Beta Testers are participants who have suitable computing experience who help make improvements to the software. The support they contribute is in addition to the professional software developers employed by the project. There are approximately 30 individuals in the Beta Testers (as of 2014). Seven members of the Beta Testers took part in the follow-up interviews. I asked these individuals specific questions about their involvement in this sub-community of participants. Three separate themes emerged from their feedback (Table 6.3)

Table 6.3 Emerging themes from Beta Testers interviews

Folding@home resource The role of the Beta Testers as a valuable resource for the project. Main tasks and responsibilities associated with membership. Motivation for involvement Why did these individuals decide to become members of the Beta Testers? Benefits of involvement ‘Perks’ or advantages to being involved in this group.

The feedback from the interviews suggest that the Beta Tester is an important resource for the project. It allows the project managers to tap into the pool of technical expertise among the folding community and to exploit the diversity of hardware and operating systems in use. Sometimes Beta Testers members have hardware that the project team

204 doesn’t have access to. This makes the Beta Testers invaluable for testing new software, and cuts down on the issues and failures when new software is released. Furthermore, this help is given voluntarily, and the Beta Testers are unpaid testers who work alongside the professional developers employed by Folding@home. One interviewee described the

Beta Testers as:

“Necessary to critical…the computer code and science need to be sorted out.” (FH1)

Another stated that many of the improvements brought about by the project team would not have been possible without the help of the Beta Testers.

“Without it, f@h would not be where it is today. We would not be able to use our hardware to the fullest.” (FH4)

Some of the individuals I interviewed had been in the Beta Testers for several years, two had been involved for over 5 years. Individuals join the Beta Testers in order to share their expertise with the rest of the community and to help out the developers in the project team.

“I am helping others with like-minded values. Everyone there is trying to help improve folding beyond just folding…I would argue that I participate far more answering questions and testing than I do actually folding. The being helpful is the more significant reason.”

(FH2)

The fact that contributing to the Beta Testers is time consuming does mean that many

Beta members do not have as much opportunity to earn points through actually folding.

The same respondent also stated:

205

“They are not personally gaining from beta testing. If points are the currency of folding, they take less because when something fails they get far less points. They are the group that cares enough about the project to make it better.”

Despite the fact that members of the Beta Testers can lose points by helping with the testing and developing work, interviewees also highlighted a number of benefits or privileges to be involved in this group. In particular, they get access to “cutting edge” software developments from Stanford, and get to see new developments before other members of the community.

“…the cutting edge aspect of it. Getting to see things beforehand is exciting.” (FH3)

“I love testing the cutting edge stuff from Stanford and being able to see what is coming before everyone else. Because after a while when you are folding the same old project it gets kinda repetitive. So every time the Pande group announces something new, I get excited.” (FH4)

Another benefit of being involved in the Beta Testers is that members get to interact more with the project team, and sometimes get a glimpse of the inner-workings of the project.

“There is more direct feedback from the scientists in PG [Pande Group] when participating in beta trials, and there is the opportunity to provide input into how details of the software can be shaped.” (FH5)

“I really like the closer interaction with the researchers. Getting some insight into the inner workings of folding and providing them with valuable feedback is very fulfilling.”(FH6)

206

Some of the Beta Testers talk of the fun and enjoyment they have had while being involved in testing.

“Building and learning about new hardware has been a blast. I hope to continue being useful to researchers and I currently plan to continue being an active folder and member of the beta team for the foreseeable future.” (FH6)

Overclockers

Eight overclockers took part in the follow-up interviews. Specific questions were about their involvement in this sub-community of participants. Three separate themes emerged from their feedback (Table 6.4)

Table 6.4 Emerging themes from overclocker interviews

An important project resource The importance of the overclocker community with regard to their technical expertise, and overall contribution. Role of this community in dissemination. Features of the overclocking Characteristics of the community e.g. helpfulness, love of technology community and competitiveness. The work within overclocking teams, cooperation and collaboration. Motivations Why do individuals become involved in overclocking and why do they contribute to Folding@home?

Much of the ‘work’ in Folding@home is completed by this sub-group of participants and their continued involvement ensures computational power for the project group at

Stanford (see Chapter Four, Section 4.2.3). In addition to this processing power, members of this community also contribute by bringing a broad range of technical skills to the project. These skills and knowledge are often shared with other enthusiasts to help them build better machines (usually via team or supplier forums), or by providing advice on what products to buy. Overclockers also share their knowledge more widely

207 with other folders on the Folding@home forum. This community also “spreads the word” about Folding@home, and individuals actively recruit others to Folding@home teams through the overclocking forums and networks, particularly when there are competitions.

“The community [overclockers] as a whole is not only spreading the word of

Folding@home through their various competitions, but by using Folding@home as a tool to measure their overclocks against, they push the work volumes for the program.” (FH8)

“There’s a few ways the community contribute. Probably the most important one is technical help, either for new contributors or longtime users…..The contests that some teams run also help the project.” (FH11)

“Overclockers / hardware enthusiasts tend to have multiple PC’s and powerful hardware.

As a result they produce a much greater amount of points and completed work units that the average F@H user. They are also the type of people that are regularly present on the forums and will help out other users.” (FH13)

The above comment refers to the helpfulness of those in the overclocking community, and this characteristic has also been referred to by several other interviewees. One respondent also described this community as “collaborative and innovative”.

“In general these are people who share common interests and are very involved in computer hardware. I like how knowledgeable most people are.” (FH11)

The generosity of this group was also referred to by one interviewee.

“People can be quite generous in a number of ways. There are members of OCAU

[Overclockers Australia] including myself who have laid out thousands of dollars in hardware for dedicated ‘folding rigs’ that just sit there 24/7 crunching numbers.” (FH15)

208

Overclocking has a highly competitive aspect to it (Bohannon, 2005), and Folding@home

(as well as other distributed computing projects) provides a way of testing an individual’s skills and knowledge.

“I see a lot of competition within the overclocking community as a whole – This is a community that naturally gathers those who love competition and / or those who are obsessed with getting the most out of a piece of equipment, much like those who tune their cars engines for maximum performance.” (FH8)

“We all share our thoughts and brag to one another about out points per day (PPD).”

(FH15)

Most of the individuals interviewed had been overclocking for several years at least.

Several spoke of the enjoyment they derive from being part of a community of like- minded individuals and of the opportunity to learn more about computer hardware. One overclocker described himself as an “eternal student of technology”. Feedback from the survey and interviews illustrate that some overclockers are drawn to Folding@home because it allows them to push their equipment and compete with other overclockers.

Observations of the overclocker forum discussions also support this. Others either initially, or after some period of time with the project, begin to see the value of the work of the project group and some interviewees have become committed to the goals of the project. One interviewee referred to this community as being in “two camps”.

“Generally, I see this group in two camps: A. Those that are folding because of their hatred of disease and wanting to eliminate it from the face of the earth. B. Those that fold because they want to build the best computer they can and tweak and twist it to get every

209 last drop of performance. I started out in camp #2…but the more I learned about what I was actually getting into, I’ve migrated to camp #1.” (FH9)

One interviewee was not aware of the science behind Folding@home when he first joined the project.

“When I first found out that the program was actually doing science and not stress testing, I looked straight into it and was amazed that we were able to simulate such things.” (FH15)

Such statements suggest that motivations in this community are dynamic and can change over time, and support the findings of Rotman et al. (2012). While the technical aspects of Folding@home may be a stimulus for initial involvement, more altruistic motivations may operate once participants begin to understand the benefits of the research (Clary et al.,1998).

6.2.2 Interviews with project team members

Two members of the project team were interviewed, including the founder of the project and an individual who has been closely involved with the project for many years. This individual has worked with both the scientists and the project developers on the software. Timing of these interviews coincided with the proposed changes to the BigAdv points scheme, and this has influenced the feedback to some extent65. Table 6.5 details the emerging themes from the interviews with the Folding@home project team.

65 The list of questions for project team members can be found in Appendix E. 210

Table 6.5: Emerging themes from interviews with Folding@home project team

The role of the project team The perceived roles, responsibilities and tasks of the Folding@home project team. Motivations of the project team Motivations that led to the establishment of Folding@home and those that sustain continued involvement. View of participant motivation Why the project team think that people take part in Folding@home and some of the perceived benefits for the volunteers. Interaction with ‘donors’ The nature of the interaction between members of the project team and project participants, and some of the issues that can arise

The role of the project team

From this feedback, it became clear that there are numerous roles within the project team. Some roles are focussed on the science, others relate to maintenance of the project software, while others are responsible for communicating with the community of active participants. The two individuals interviewed have quite different roles within the project. The founder of Folding@home wrote much of the initial project software, and is now responsible for the overall management of the project.

“I’ve done about everything. In the beginning, I managed and wrote some of the code.

Now it’s management of the project as a whole.” (FHPT 1)

When asked what the main role of the project team is, or should be, he stated that it was:

“To do the best science we can. People are donating a vast resource to us and in the end, we will be judged by the impact of our scientific output.” (FHPT 1)

211

The other interviewee is now one of the dedicated moderators on the Folding@home forum, and also plays a role in addressing and prioritising software bugs that gets identified by the community.

“I consider myself an expert in all things that can go wrong from the perspective of the general public (donor-community). (FHPT 2)

This individual is also involved with much of the day-to-day interaction with project participants and provides an interface between the project team and the community of participants.

“I have important abilities that help to bridge the gaps between Donors and Scientists. I spend several hours most days along with similar support from other Moderators /

Administrators.” (FHPT2)

There was an awareness of the diversity of the roles in the project, but also an acknowledgement that some roles do not always get the time they require, such as advertising the project.

Motivations of the project team

Folding@home was set up as an “ambitious scientific research project”. It requires significant computing power, and these needs are best addressed through a distributed computing framework.

“We realized that there was an opportunity to do something grand. We had the algorithmic ideas in place and needed the raw horse power to push it forward’” (FHPT 1)

This individual also spoke about the research output of the project, and that his work had been “fundamentally advanced”.

212

The second interviewee also spoke about the research output of the project, and that the impact of this work was something that motivated his continued involvement in the project.

“It’s truly dedicated to the betterment of mankind (specifically the health of mankind)”

(FHPT 2).

He also enjoyed the intellectual challenge of his involvement, and some of the friendships that he made as a result.

“It satisfies one personal goal: to be mentally challenged to learn more and to figure things out. It satisfies another personal goal, to do good for mankind. I’ve developed a number of good friendships, though that wasn’t part of my original goals.” (FHPT 2)

This response demonstrates that other aspects of participation (e.g. forming friendships) can arise or develop over time, and may not have played a role in motivating their initial involvement in the project.

View of participant motivation

The interviewees were asked directly why they thought people became involved in

Folding@home. Both felt that there were numerous motivations in operation.

“For some it is the science – our goals and the ability to make a contribution. For some, it is the scale of the whole project and the interest in computers and what can be done with them.” FHPT 1

The above makes reference to the community of overclockers and hardware enthusiasts.

This individual also acknowledged the social aspect of participation, and that this may important for some.

213

“I think the ability to connect, whether that be to make a difference, to do science, or to connect with other people – these are all important areas.” (FHPT 1)

The fact that many participants had been affected by the disease being researched by the

Folding@home scientists was referred to.

“Most everyone has known someone who has been touched by the diseases being studied.” (FHPT 2)

The same interviewee also stated that the fact that the project had resulted in a number of publications and was not profit driven may also be important to participants.

“The multitude of publications prove that it is good science. It’s not driven by a profit- oriented company but contributes to the open uses of scientific advancements.” (FHPT 2)

These responses corroborate much of the feedback on motivation provided by participants who took part in the online survey and interviews.

Interaction with ‘donors’

The final theme to be identified in the interview feedback relates to how members of the project team interact with Folding@home participants (often referred to as ‘donors’).

Issues that can arise as a result of these interactions were also identified.

The founder of the project stated that they aim to get ‘involved and engaged’ with the community of participants. However, he also added:

“…there are so many demands on my time, and there are so many donors to connect with.” (FHPT 1)

This sentiment has been expressed by most of the project scientists who have been interviewed, as they try to balance the demands of carrying out research, with the 214 demands of managing large numbers of citizen scientist volunteers. Apart from these statements, this interviewee was reluctant to discuss interaction with participants in any great detail. Given the level of hostility directed towards the project team (and this individual in particular) observed on the Folding@home forum and several of the other folding forums in the wake of the changes made to the BigAdv points system, his reluctance to discuss these topics is perhaps understandable. The controversy surrounding BigAdv has highlighted some of the problems associated with working with a large community of volunteers, many of whom feel that their contribution to the project is expressed in terms of the points awarded to them. Some participants felt undervalued as a result of these changes, and by the fact that they were not consulted beforehand.

The other interviewee who is a forum moderator is involved to a much greater extent in interactions with participants.

“We maintain a forum environment that is conducive to constructive discussion, fight forum spam etc. in a way that properly fits the title of Moderator / Administrator.” (FHPT

This role of ‘interface’ between the project team and the participants is important in such a large community, and there is some recognition that the scientists don’t always have the time to do this.

“Donors need to know that somebody is listening to them and supporting them, some of which I can do, a role that should not be overly burdensome to the scientists.” (FHPT 2)

When asked about interaction with the folding community, he stated that most of his interactions were brief and tended to focus on a specific problem. There are some challenges however.

215

“Some [participants] are very knowledgeable and some are not. In brief interactions, it’s difficult to know which. It’s easy to incorrectly assume my audience is at one level when they’re really at a different level, making communications difficult.”

This comment may help to explain the few negative comments that were made by participants in the online survey about the official Folding@home forum, and highlight some of the difficulties in managing asynchronous communication (James and Busher,

2009a, Abawayj 2012).

6.3 Summary of Results

The feedback from the survey illustrates that this group of respondents is almost entirely male and mainly aged under 40. They are generally well-educated with a background interest in science and technology. Over a third (150 individuals) are in IT-related professions, and many (80) are currently students. This survey has highlighted the importance of an active community of overclockers and hardware enthusiasts, and has helped to address RQ1 (Who participates in online citizen science?).

This group of active participants interacts online through Folding@home, team forums, and on forums that are linked to groups of overclockers and hardware enthusiasts. The longer a participant has been with the project, the longer they spend participating in online forums and discussions. These observations have helped to address RQ 4 (How and why do project participants interact online?) and RQ5 (How can contribution to online citizen science projects be characterised?).

Over 200 respondents are motivated to take part in Folding@home so that they can make a contribution, either to science, or to a ‘worthy’ cause, and 74 respondents report that they, or someone close to them, has been affected by one of the diseases being

216 investigated by the project. One of the most commonly cited motivations for taking part is to fully utilise computing power and this is indicative of the involvement of the sub- community of overclockers and hardware enthusiasts (Bohannon, 2005). Respondents maintain their involvement with the project because they like making a contribution, and because getting involved is relatively straightforward. This feedback has helped to address RQ 2 (What motivations initiate and sustain participation in online citizen science projects?), and also highlights the importance of making a contribution when considered within the context of previous research (Holohan and Garg, 2005, Raddick et al., 2010,

Krebs, 2010, World Community Grid, 2013).

Over a third of respondents (35%, 138 individuals) feel they are making a contribution to science, while just over 100 respondents felt they were making only a small contribution, or enabling the work of the scientists. Over 50 respondents felt that they were just making a donation of their computing power, similar to a cash donation to a charity. This feedback has helped to address RQ 6 (How do participants perceive their role in the project?).

Interviews with overclockers, and members of the Beta Testers has revealed that these groups of participants are important contributors to the project in terms of their computing output, and level of technical expertise they bring to the project (RQ 5). Some overclockers invest significant amounts of money in their computers and in running them which is of great benefit to Folding@home. Both of these communities see their contribution as crucial to the project (RQ 6), and most report deriving enjoyment through their involvement in these groups (RQ 2). However, there were some significant issues highlighted in these interviews relating to their relationship and perceived treatment by the project scientists (RQ 4

217

Interviews with two members of the project team reveal that advancing the science of protein folding and accomplishing “something grand” were the main motivators to setting up Folding@home (RQ 2). For one member of the project team, being able to contribute to improvements in human health were important, as were developing friendships and good working relationships with others. There was an acknowledgement that the efforts of participants have significantly advanced this area of science. However, recent issues associated with changes in the way points are awarded to high-level participants has clearly had an impact on the relationship with the community of participants. Managing a community of many hundreds or thousands of participants has been challenging for the project team during the time when much of this data was collected.

This data, along with my own experience as a participant demonstrates that in order to make a significant contribution to this project, a degree of technical knowledge is important. The community of overclockers have made a substantial contribution to

Folding@home, and there is evidence on the numerous forums that participants are committed to the long-term project goals, and will readily offer support to other members of the community. The changes relating to BigAdv have provided an insight into the relationship between the project team and the community of active participants, and highlight the importance of participants feeling valued by the project team.

218

Chapter 7: Planet Hunters Results This chapter presents the findings from the Planet Hunters survey (see Appendix D) and a thematic analysis of the interviews with 18 citizen scientists and four members of the project team.

7.1 Online survey results

In total, 118 individuals responded to this survey. Approximately 160 000 individuals have registered with Planet Hunters. However, the number of participants actively contributing to the project is estimated to be in the region of 300 (see Chapter Four,

Section 4.3.2). The results have been broken down into four main areas: demographic characteristics; patterns of participation; motivation and reward; and interaction and contribution. These correspond to the areas being investigated and to the focus of the research questions. With each result, n=118 unless otherwise stated. In the bar graphs, the number of participants in a specific category or providing a response is on the top of each bar.

7.1.1 Demographic characteristics

The majority of respondents to the survey were male (74%, 87 individuals), with a relatively equitable distribution among the different age groups. Most respondents were well educated with 65% (77) having either an undergraduate or postgraduate degree

(Figure 7.1). Most of these qualifications (75%) were in a STEM subject.

219

Figure 7.1 Highest level of educational attainment

still studying other 2% 2%

high school / GSCE postgraduate 18% UK A levels / degree BTEC 24% 7% Junior undergraduate college/HND degree 6% 41%

Most participants are from developed countries. Nearly half (57) were based in Europe,

43 were in the USA and Canada, and 10 were based in Australia, New Zealand or South

Africa. Other countries represented included India, Japan, Argentina and Venezuela.

Table 7.1 illustrates the professions represented by this sample of Planet Hunters participants. There is a relatively high representation of IT professionals (14), business professionals (13) and students (12). Eleven respondents work in the sciences.

220

Table 7.1 Profession of respondents

IT / software 14 professional Business professional 13 Student 12 Work in the sciences66 11 Engineer 11 Skilled trade 8 Educator 6 Administrator 6 Retired 6 Artist / creative 6 Unemployed 4 Sales 2 Civil Servant 2 Librarian 2 Leisure industry 2 Medical profession 2 Police 2 Retail 1 Home-maker 1 Gardner 1 Hospitality sector 1 Pilot 1 Translator 1

Again, one of the most notable features of the demographic data is the predominance of male respondents. According to one of the Zooniverse scientists, a skew towards male participants is found in a number of Zooniverse projects (Lintott, 2014)67. Furthermore, astronomy is a subject that has been male dominated both in the professional and amateur arena68. According to a report published for the Royal Astronomical Society, only 28% of UK university lecturers in astronomy and 7% of astronomy professors are women (McWhinnie, 2011). The largest UK amateur astronomy association, The British

Astronomical Society, estimates that fewer than 10% of their members are women

66 Survey responses included professions such as lab technician, meteorologist, physicist, chemist, scientific researcher, science writer, geologist, and “scientist”. 67 Zooniverse users do not provide any personal data such as age or gender. Zooniverse scientists have estimated the proportion of male and female participants by looking at the names on email addresses. 68 Statistics relating to women in professional astronomy have been compiled by the International Astronomical Union (IAU): http://iauwomeninastronomy.org/statistics/international-stats/. 221

(Bowdley, 2009). Given these figures, it is perhaps unsurprising to find more male than female participants in an astronomy-based citizen science project such as Planet Hunters.

7.1.2 Background interest in science

Almost this entire group had taken part in other science-related activity during the past year. Figure 7.2 illustrates the types of activities and the numbers of respondents reporting taking part in them during the past year.

Figure 7.2 Other science-related activities undertaken during the last 12 months

105 103

81 75 69 69 56

17 15 5 1

Perhaps not surprisingly, given the subject of the Planet Hunters project, over 80 individuals (68%) had taken part in an amateur astronomy activity during the past year.

In addition to their participation in Planet Hunters, many survey respondents (80, 68%) report participating in other citizen science projects (Figure 7.3). Half (59) have taken part in other Zooniverse projects such as Galaxy Zoo, Old Weather, Moon Zoo and Planet

222

4, while a smaller proportion (20%, 24 respondents) have taken part in distributed computing projects (mainly SETI@home).

Figure 7.3: Other citizen science projects that respondents have participated in (n=93).

5 2 2 1 3

Just under a third of respondents reported that they first heard about Planet Hunters via the Zooniverse. Most respondents had first heard about Planet Hunters via an online article or link.

7.1.3 Patterns of participation

A number of survey questions were posed to ascertain how individuals participate in the project. For example, I asked how many hours a week they spent on Planet Hunters, and whether they contributed to the online community. Many of the respondents had been with the project for some time, with 60% (71) participating in the project for more than one year. Figure 7.4 illustrates how much time they spent (on average) per week taking part in Planet Hunters.

223

Figure 7.4 Number of hours respondents spend participating in PH per week (on average)

5-7 hrs 11-15 hrs 5% 2%

2-4 hrs 18%

< 2 hrs 75%

Seventy-five per cent of respondents (88) report spending fewer than 2 hours per week classifying light curves. A much smaller number of individuals (8) participate for more than 5 hours a week. According to the Zooniverse Team, many Planet Hunters have a casual pattern of participation. Participants often classify light curves when they have a bit of spare time, and the average time of a participant session (based on user logs) in

Planet Hunters is 11.9 minutes (Simpson, 2013).

Forty-nine respondents (41.5%) do not participate in the online discussions in any way, meaning that they neither read or post content, and focus on classifying light curves.

This included posting content via the Talk function, and the online discussion forum.

Forty-three respondents (36%) stated that they read content only, while only 26 (22%) within this group had ever posted content. According to one of the Zooniverse scientists, the number of people posting content in this group is a much higher percentage than they would generally expect (Lintott, 2013). Data provided by the Zooniverse team on the number of Talk comments posted on Planet Hunters, suggest that nearly a third of the total number Talk comments have been posted by only 48 participants (Raddick, 2013a).

224

This is a similar finding to that of Tinati et al. (2014) who also found that small numbers of participants contribute to online discussions in other Zooniverse projects.

For those respondents that do take part on the project discussion forums (either reading or posting), 90% report spending less than 1 hour a week there, while a very small number of individuals report spending more than 2 hours a week on the discussion forums. This small group also report spending more time participating in Planet Hunters per week, and may well number among the project’s core group (See Chapter Four,

Section 4.3.2).

7.1.4 Motivation and reward

Questions relating to motivation were open-ended, and content analysis was used to categorise the responses into important themes. This process has been described in greater detail in Chapter Three (Section 3.4.1). I asked respondents why they decided to try Planet Hunters and the responses were coded into 13 different categories illustrated in Figure 7.5.

Figure 7.5 Why respondents decided to try Planet Hunters

11 10 9 7 6 6 6 5 3

225

Most respondents gave more than one answer to this question, but the most commonly cited reasons for trying Planet Hunters (47%, 56 respondents) was a background interest in the science. After a prior interest in the science, the next most popular reason for trying Planet Hunters was to make a contribution to scientific research (34%, 40 respondents). These motivations were also observed to be of importance in previous studies of Galaxy Zoo participants (Raddick et al., 2010, Raddick et al., 2013), although this observation did not inform my own coding scheme.

The possibility of making a discovery was a big attraction for 26 respondents, and seven individuals in this group of respondents have been involved in the discoveries of Planet

PH1b and Planet PH2b (Schwamb et al., 2013, Wang et al., 2013). For a smaller group of respondents (6%, 7 individuals), the fact that Planet Hunters is a Zooniverse project was important. Many respondents take part in other Zooniverse projects, and there are similarities in the layout and format which may make Planet Hunters feel more familiar to some participants.

Only 5 (4%) respondents stated that the opportunity to learn was one of the reasons that they began to participate in Planet Hunters, although the desire to learn more about exoplanets may be part of the ‘interest in the science’ motivation category. In order to find out what kept participants involved in the project, they were asked what they liked best about Planet Hunters (Figure 7.6).

226

Figure 7.6 What respondents liked best about the project

29 21

11 9 9 8 4 4 2 1

Again, being able to make a contribution to science was one of the most important features of Planet Hunters for over a third (39) of respondents, and one of the things that kept them contributing to the project. The possibility that a new discovery may be around the corner continues to remain an important motivator for about a quarter of respondents (29). The accessibility of the project and the user-friendly interface are also important features of the project for over a fifth (21) of respondents. The concept of a

‘community‘ surfaces in the responses to this question, and 11 respondents (9.3%) stated that the Planet Hunters community was one of the things they liked best about participating.

In order to gain a further insight into participant motivation, respondents were asked whether they thought Planet Hunter participants should be rewarded for their participation in some way (Figure 7.7).

227

Figure 7.7 Do you think you should be rewarded for your participation in Planet Hunters? (n=110)

yes 16% don't know yes -general 5% recognition for contributions or discovery no, 42% participation is enough 37%

Many participants (42%, 46 individuals) wanted to be recognised for any discovery they might make, which they already are through co-authorship and acknowledgement on scientific publications (Wang et al., 2013, Schwamb et al., 2013, Schmitt et al.,2014).

Over a third of respondents (41) felt that participation was its own reward, and that adding an extra incentive would change the nature of the project.

A minority of respondents (16%, 19 individuals) felt that an extra reward may lead to a greater numbers of participants, or that those who contribute more than other participants should receive something extra. A number of suggestions were made including t-shirts, vouchers, certificates, ‘virtual’ badges, discounts and lab tours.

7.1.5 Interaction and contribution

Respondents were asked to describe their interactions with other participants through an open-ended question. Sixty-seven respondents answered and their answers were coded into 3 separate categories which related to whether they read content or posted content also (Figure 7.8). 228

Figure 7.8 How would you describe your interaction with other participants? (n=67)

some interaction (incl posting) 30% no interaction 52%

minimal interaction (read only) 18%

The lower response rate for this question could have been due to the fact that the remaining 55 respondents do not interact with other participants in any way. In the previous question relating to how much time they spent on the online discussion per week, it was evident that nearly half of respondents do not spend any time at all on the project Talk or discussion boards. Of the 67 that answered this question, 35 reported no interaction at all with other participants. Twelve respondents stated that they only read the content on the online discussion, so again, had minimal interaction with other participants. Only twenty individuals actually post content, and respond to the comments and/or posts of others. This roughly corresponds with the feedback relating to how much time respondents spent on Talk or the discussion boards.

The majority of respondents to the survey appear to focus on analysing light curves, and do not actively take part in the online Talk or discussion boards. One respondent made the following comment:

229

“I have had no interaction with other participants. In fact, with the way the site is currently set up, unless you go looking for other people, you wouldn’t know that there was anyone else around,”

Discussions via the Talk function are only prompted after a participant has classified a light curve, which may not appeal to the casual participant who does not wish to discuss individual objects in detail (see Chapter Four, Section 4.3.3). Furthermore, a participant can only access the more general discussion boards by going into the Talk function, so unless a participant has used the Talk function before, they may not be aware of the presence of these more general discussion boards .

View of contribution

How participants view their own contribution to the project was an issue that was highlighted in the follow-up interviews with Foldit players, and I was interested in exploring this with Planet Hunters participants also. The online survey contained the question: ‘How would you describe your contribution to Planet Hunters? Do you feel as though you are involved in scientific research?’

All 118 respondents provided an answer to this question and most (82%, 97 individuals) felt they were involved in scientific research to some degree. For the majority of these respondents this contribution was felt to be quite small. However, there was a sense of respondents feeling part of a wider, group effort. Those who had been involved in the discovery of a new exoplanet (7 of the respondents) expressed a sense of pride at having helped in these discoveries.

For most of the respondents, Planet Hunters does not necessarily make them feel at the forefront of scientific research, but more as helpers. One respondent likened the role of

230 participants to lab assistants. This differentiation between the analysis of light curves by citizen scientists, and the further evaluation of that data by professional scientists may relate to views and beliefs of the respondents about what actually constitutes science and scientific research (i.e. basic classification tasks are not as important as more in-depth analyses).

7.2 Interview feedback

The results of the interviews have been subjected to a thematic analysis using the approach of Braun and Clarke (2006) which was outlined in Chapter Three (Section 3.4.2).

The major themes will be presented for both the interviews with citizen scientists (n=18) and with members of the project team (n=4). The emerging themes will be outlined.

7.2.1 Interviews with citizen scientists

Table 7.2 lists the emerging themes from the interviews with the citizen scientists

Table 7.2 Emerging themes from interviews with citizen scientists

Interest in astronomy Prior interest in astronomy, including formal and informal study, involvement in amateur astronomy, and communication activities relating to astronomy. Science fulfilment The desire to be more connected to science and the opportunities Planet Hunters presents to get involved in scientific research. What I like about Planet Hunters Aspects / characteristics of the project that were admired by respondents, and perhaps motivate their involvement in the project. What have I learned? What participants have learned about the science behind planet hunting, as well as learning about citizen science projects more generally. Contribution and participation How respondents view their efforts in the project. What respondents feel about the tasks they are asked to perform and how active respondents are in the project. Getting more involved in the project, and in data analysis. Interaction and communication How respondents interact (or do not interact) with other project participants, and with the project scientists. Views regarding the project interface.

231

Interest in astronomy

One of the most common themes to emerge from the interview data, and one that was expressed by nearly all of the interviewees was a prior interest in astronomy. Most of the respondents stated that they had had an interest in astronomy since childhood.

“I have been interested in astronomy since I was a young kid and that interest remains to this day.” (PH2)

“There was always at the back of my mind a basic interest in the stars and planets and a curiosity about all things of that type.” (PH3)

Not only have most of those interviewed expressed an interest in astronomy, but just over half have also stated that they have some involvement in amateur astronomy either through their membership with local amateur groups, or making observations with their own telescopes. Several respondents even report conducting their own research in astronomy.

“I am an amateur astronomer and have always been interested in our solar system in particular. I have even written a few blogs in regards to astronomy, especially to put some flawed ideas and theories to rest by doing a lot of research on my own.” (PH1)

One participant used their involvement in Planet Hunters as a way to further their own independent research in astronomy.

“..to make contact with those who have skills I’d like to tap into for my own research (esp. statistical analyses).” (PH5)

Most respondents report reading about astronomy online (particularly though the NASA website or Astronomy Picture of the Day) and through the news and magazine articles.

232

About a third of respondents, report following key projects and scientists through social media sites such as Facebook and Twitter. Over half of respondents reported taking part in other astronomy-related citizen science projects – particularly other astronomy projects in the Zooniverse such as Galaxy Zoo, Space Warps, Moon Zoo and the

Andromeda Project, and other online astronomy projects such as CosmoQuest (a classification-based project that maps craters on the moon and on Mercury) and

SETI@home.

Two respondents stated that they had some formal qualifications in astronomy. One of these had a BSc in astrophysics and was also a fellow of the Royal Astronomical Society

(the learned body for astronomers in the UK), although employed as a software engineer.

The other respondent had returned to formal education and was studying an astronomy module as part of an undergraduate science degree. A few other respondents had undergraduate degrees in physics or mathematics. However, none of the interview respondents was a professional astronomer, or worked in a related scientific discipline.

Science fulfilment

As well as an interest and involvement in astronomy to varying degrees, most of the respondents expressed a wider interest in science. Indeed, over half of the interviewees have a formal undergraduate-level qualification in science in areas such as physics, oceanography, medical sciences, computer science, chemistry and geology. Despite a general interest in science and formal academic qualifications, none of the interviewees report working in scientific research or in a scientific field. Several have in fact, expressed regret at not being in a science-related occupation despite initially training for a career in science.

233

“it is sad to think that the time spent learning at school and university is generally wasted by many people like me who are forced into other fields and occupations. I would like to do more in the sciences, and I think this is a great way to start.” (PH10)

Participating in projects like Planet Hunters therefore provides a way for those with an educational qualification, or interest in science who work in unrelated professions, to reconnect, or remain connected, with science in some way. Participation in online citizen science projects may help to fulfil a need to maintain this connection with science, and many of these individuals used the interviews to express gratitude to the project organisers for providing them with this opportunity.

One respondent left school at an early age with no formal qualifications, and stumbled upon Planet Hunters during retirement. With growing experience and confidence, this respondent eventually co-discovered a new exoplanet.

“Working on this project has really stretched my horizons….it has been great fun and given great satisfaction, the fact that I have been credited with the discovery of a new planet added to the experience.” (PH3)

Participation in projects like Planet Hunters also provides this experience for those with an interest in science, who may have no, or few, formal educational qualifications in the subject.

What I like about Planet Hunters

Respondents generally do not refer to motivations as such, but talk more generally about what they like, or what they find appealing about Planet Hunters. Other motivations may be referred to more indirectly, or in their responses to some of the other questions. The feedback from the interviews generally agreed with the results obtained in the online

234 survey in the questions relating to motivation and reward. For example, most of the interviewees were drawn to Planet Hunters because of an interest in astronomy, or a more general interest in science. A few of the respondents liked the fact that they are able to make a meaningful contribution to scientific research in some way.

“…I keep participating because it is good to be actively taking part in science.” (PH2)

“Love astronomy and science and it’s great to get the chance to be part of some real science.” (PH11).

One of the most appealing aspects of Planet Hunters for those interviewed (as in the survey) was the possibility of discovering a new exoplanet, and being the first to make such a discovery. Three of the interviewees had been involved in the discovery of an exoplanet.

“Looking for new planets was appealing in that it is such an exciting thing to be part of and perhaps be one of many who can have their name associated with a new find.” (PH2)

“I kept participating in Planet Hunters because I knew that in order to increase my odds of finding an exoplanet I would have to go through a lot of data.” (PH14)

“I thought what a cool job that is, to hunt for planets…..and immediately understood the quest.” (PH1)

About a third (six) of those interviewed stated their admiration for the goals of the project, and that they liked the idea of breaking down, or crowdsourcing, the task.

“There are lots of projects around and we’ve barely scratched the surface in getting people involved. The idea that you can take a huge problem and break it into a billion tiny pieces is pretty neat.” (PH8)

235

“A great way to crowdsource data – and people like to feel involved in large projects even if they don’t have the aptitude to be in that science field but have an interest.” (PH12)

Five interviewees stated that taking part in the project was fun or enjoyable, and one individual stated that taking part provided an ‘escape’. Another stated that it helped them to unwind after a stressful day at work.

What have I learned?

Interview participants were asked what they had learned through their involvement in

Planet Hunters. Most of the answers to this question outlined what the respondents had learned about the science associated with extra-solar planets, supporting previous findings that participation in citizen science can play a role in informal science learning

(Evans et al., 2005, Jordan et al., 2012, Jordan et al., 2011, Price and Lee, 2013). However, none of the respondents stated that the opportunity for learning was what they liked about taking part in Planet Hunters, or indeed, cited it as a reason for joining or remaining with the project in their feedback to the other interview questions.

There is a suggestion that some respondents may have learnt a bit more about the scientific research process. The tasks associated with Planet Hunters could be considered representative of ‘typical’ scientific research. For example, in order to make a discovery a lot of data must be processed, which at times, can be quite repetitive and may take many months.

“To do well, they require a dedication of time and effort to grasp not only the tools but also the concepts.” (PH5)

“Reminded me about the grind involved in checking samples and collecting data!” (PH6)

236

Those involved in exoplanet discoveries may get more of an insight into the various stages in scientific research and in dissemination of results. One respondent who had been involved in the discovery of an exoplanet talked about her experience sharing her findings with others and how she discussed her potential discovery with other project participants. This may have demonstrated to her the importance of verification, and of seeking out the opinion of one’s colleagues or peers. A further insight for those involved in discoveries may involve an increased appreciation for the ‘end-products’ of science, and the collaborative nature of the effort required to produce results, and to produce publications – knowing that they are one of many involved in a single discovery.

“I have already been credited with the discovery of a new planet…It was a real thrill to know that I was one of only about 40 non-professional astronomers in the world to be credited with that distinction.” (PH3)

Not only have some participants been acknowledged in publications or co-authors in papers, but some have found themselves talking to journalists about their experiences, as well as to other scientists and science communicators. This has given them direct experience of communicating about scientific research.

“I was the first one to flag up a possible exoplanet candidate. She [Meg Schwamb, member of the project team] also asked me if I was available for an interview….Stephen

Craft was writing a piece for Scope (an online publication at MIT) on my recent success….Then post-graduate students contacted me in relation to a documentary film

237 project. The Royal Observatory Edinburgh and the Physics department St Andrews commissioned a film for the Edinburgh International Science Festival this year69.” (PH1)

“And I also had my 15 minutes of fame when a quote of me in The Guardian got picked up by the Belgian press, and I got inundated by phone calls and interviews for a day! My confidence in speaking to the press has obviously increased as well. And also, since I am not a native speaker, my English has improved a lot.” (PH13)

Clearly, the majority of Planet Hunters participants do not get this opportunity, and the three interview respondents who have made a discovery have classified thousands of light curves each, and have been involved with the project for some time.

Contribution and participation

The responses relating to how participants view the task, as well as responses regarding how much time they spend on Planet Hunters in comparison to other activities, have helped to build up a picture of how this group of participants view their contribution and what they think of the tasks. The task itself is perceived by most respondents to be straightforward if a little repetitive.

“I think they are simple tasks, they don’t take long and can be easily done if focused.

Watching the light curves can be tedious but the fact that there is something out there waiting to be discovered is very rewarding.” (PH1)

“The tasks are easy enough and not over taxing.” (PH2)

69 This film ‘Close Distance’ can be viewed here: http://www.roe.ac.uk/vc/content/wywhexoplanets/close- distance.html. 238

A few respondents have stated that they would have liked more feedback from the project scientists while they were learning to classify light curves, and stated that at times they were not confident that they were classifying the light curves correctly.

“…I feel like there was not enough examples and tutorial / training – I often wasn’t certain what I was looking for or doing and I note that many other participants felt the same way in the community forum.” (PH12)

“Sometimes I’m nervous when I’m not sure about the light curves being right.” (PH15)

Three respondents expressed a desire to go further in the project and to get more involved in the analysis of the data. There are some basic tools in Planet Hunters that allow participants to more closely examine the light curves, and some committed and more technically skilled participants have shared resources for conducting certain types of further analysis, for example, using the raw data to calculate planetary diameter 70.

Two of the respondents mention carrying out their own research in astronomy, although neither specified whether this was using the data from the Planet Hunters project. One respondent talked of her desire to actively encourage other project participants to become more involved in the project.

“I also have an interest in fostering, encouraging, etc. as many of my fellow Zooites to go beyond classifying: get involved with data analysis, testing hypotheses etc.” (PH5)

This respondent also actively seeks out fellow participants who can help her with her own research using the Planet Hunter data.

70 This page on the Planet Hunters Discussion Board provides links to tools for further examination of the project (Kepler) data that have been created by project participants (accessed 21/04/14). http://talk.planethunters.org/discussions/DPH1014lr5?sub_board_id=planet_hunters_resource_library 239

The amount of time interviewees spent on Planet Hunters varied greatly. Four stated that do not currently participate due to lack of time, while three stated that they contribute for approximately half an hour a week. Only a couple of interviewees stated that they spend a bit longer participating (1-2 hours per week). Only one respondent described Planet Hunters as their main interests, and that they spent as much time as they could classifying light curves. Three stated that their participation was seasonal – either related to work schedule, or that they participated in the summer months more, when it was harder to look through their telescopes.

One respondent stated that he moved from project to project and may not participate in

Planet Hunters all the time.

“I also only analyse data for a month or two before becoming interested in some other project. With so many amazing citizen science projects available online it is difficult to find time to participate in them all. I tend to only participate in the projects I feel I can make the greatest contribution to.” (PH14)

Just over half (10) of the interviewees spread their free time among a number of different projects on the Zooniverse collection of projects is of interest. Indeed, that is what some find so appealing about the Zooniverse projects. Despite the fact that some of the interviewees only participate lightly (or not at all) in Planet Hunters, or share their attention among several different projects, half (9) of the respondents stated that they felt the work they did on the project was in some way useful or beneficial to scientists.

“The contribution a volunteer ‘citizen scientist’ can make to research is huge and we are all well loved by those in charge of the project.” (PH1)

240

“I think that they [the project tasks] are interesting, useful and of some great value to scientists who use our findings.” (PH3)

The utility of the task assigned to citizen scientists is believed to be an important motivator by the Zooniverse scientists, and they have stated that it is important not to waste the time of those who participate in the Zooniverse projects71.

Interaction and communication

Interviewees were asked about their interaction with other project participants (including the project scientists), and what type of interaction was expected. Insights about interaction and communication were also gained from the responses to some of the other questions. Of the eighteen respondents, half have stated that they do not interact at all with other participants.

“I don’t participate in the community, but I know it is there. I expected a community forum of some kind for guidance and discussion.” (PH12)

Two of the respondents act as moderators on other Zooniverse projects, and so are more familiar with a number of the scientists, and with some of the more active participants both from Planet Hunters and from other Zooniverse projects.

“Since I’ve been very involved in the various Zooniverse projects, I already ‘know’ some of the scientists or moderators and I recognize user-id’s from other projects. So for me, this feels very comfortable.” (PH13)

These moderators were also somewhat critical of the Talk function in Planet Hunters, and the general set-up of the Planet Hunters online community, which is different to some of

71 The Citizen Science Alliance that oversees the Zooniverse states in a brief guide that projects should not waste participant’s time http://www.citizensciencealliance.org/downloads/zooniverse_guide.pdf. 241 the other Zooniverse projects which have a more ‘traditional’ online forum set-up (such as Galaxy Zoo or Space Warps).

“The only improvements I’d like to see are improvements in Talk (the ‘forum’ developed by the Zooniverse). As I understand it this is really helpful for the scientists, but for me as a user it lacks the easy use we had with the old fashioned forum on the first GalaxyZoo. I am a moderator on a couple of new Zooniverse projects that use Talk and we never have the same feel of a community there.” (PH13)

Another respondent also made a comment relating to the tools available for interaction.

“The time I want interaction is when I find an image I can’t figure out in the way desired.

There are facilities for this on the sites but they are a bit cumbersome. They do not lead to group analysis really.” (PH4)

There were very few comments relating specifically to interaction with the project scientists. One very active Zooite stated:

“..after much strong involvement, project scientists have largely disappeared from the other astronomy Zooniverse projects….and I was pleasantly surprised to see how active

Meg is (no sign of others though).” (PH5)

One of the respondents who was no longer very active in Planet Hunters but had become active in the citizen science game EteRNA, made several comments about what he thought project scientists should be doing in their interactions with citizen scientists.

“I think interaction with project scientists is very important. The EteRNA project has bi- weekly group chats with the developers and scientists and players have been able to make significant contributions to the functionality of the website and project. I think regular

242 interaction with project scientists helps participants stay motivated because it makes them feel like they aren’t working in vain and that they are important to the project.”

(PH14)

When talking about the project tasks, this same individual stated:

“If participants who analysed more data were somehow rewarded by getting more recognition or interaction with project scientists I think it would help retain those who are the most dedicated.”

From these interviews it appears that the overall interaction between the participants, and between the participants and project scientists is somewhat limited, unless that respondent has been involved in a discovery, or has been a moderator on one of the other Zooniverse projects. About a quarter of the respondents (4) were slightly critical

(either directly or indirectly) of the current Planet Hunter interface for interaction and communication (the Talk function) and there is a sense that they would like more feedback from the scientists, particularly when it comes to uncertainties surrounding the project tasks.

7.2.2 Interviews with the project team

Five major themes emerged from the interview feedback from the Planet Hunters project scientists. These are outlined in Table 7.3.

243

Table 7.3 Emerging themes from interviews with project scientists

Benefits for scientists The positive aspects and outcomes that are a direct result of their participation in the project. The motivation of the scientists involved. Role of the project team The perceived roles and responsibilities of the project team in managing Planet Hunters. View of the volunteers’ motivations Why the scientific team think that people take part in Planet Hunters, and the perceived benefits for the volunteers. Varying levels of involvement The recognition that there are different levels of interest and engagement with the project. The acknowledgement of the contribution of core participants. Working with the participant Some of the responsibilities and challenges associated with working community with a large online community of volunteers.

Benefits for scientists

One of the main benefits for the Planet Hunters scientists is that they get help with research that relies on human observation. The tasks on Planet Hunters cannot be carried out with total reliability by a computer, and human participants have identified exoplanet candidates that have not been detected by the Kepler computer algorithms.

“It’s science that you can do with the public that can’t be done by your computer algorithms.” (PHPT1)

The involvement of thousands of volunteers also enables very large datasets to be analysed, and such datasets are becoming increasingly common in astronomy. One scientist talked of the difficulty in keeping up with all of the data output produced by

Planet Hunter participants. One scientist spoke of citizen science as a new research tool, one that could complement other ways of doing research.

244

“I would say it’s transformed how I do my research. Citizen science is very much a tool we can use for large datasets, particularly in astronomy – and we’re heading towards more data.” (PHPT2)

From the scientists’ responses, it appears that the primary motivation to participate in online citizen science projects like Planet Hunters, is that it provides an effective way to carry out, or ‘crowdsource’ a relatively straightforward analytical task that requires little training or supervision. Time that would have been spent by scientists on these tasks is now freed up for more in-depth analysis of the data. However, this is not the only motivation for their involvement, and it is evident that for most of the scientists involved, the opportunity for public engagement and outreach is also appealing.

“I think it’s an interesting way to do outreach, I think it brings people into the scientific process and the actual organic component of it…” (PHPT2)

When this scientist described outreach activities, it became clear that outreach for her involved telling people about her research and educating them about some of the underlying science.

“We talk about what science we want to do, and how do we do that, and how do we educate the volunteers…… My goal when being on Talk has been to make sure that information is being disseminated properly.” (PHPT2)

This scientist also saw Planet Hunters as enabling outreach at the same time as she carried out her research thus ‘killing two birds with one stone’. It also makes her consider different avenues of communication.

245

“…it’s made me more creative in how I do outreach. I can satiate doing outreach as well.

Having live chats and things like that. We don’t need to do it, but I think we should do it….is there something new I can do with Planet Hunters volunteers….?” (PHPT2)

It is evident that the scientists get some enjoyment out of their participation.

“It’s fun! Both in terms of engagement and in terms of research.” (PHPT3)

This same scientist also spoke of the enjoyment associated with the intellectual leadership involved in managing the project.

Role of the project team

One of their primary and underlying roles involves making sure that the project site is well maintained, and functioning. This requires that members of the project team liaise on occasion with members of the Zooniverse team, particularly those involved in the more technical aspects of the projects. It also requires that members of the project team have a presence on the project site, and are aware when there is a technical issue.

“If something’s broken, I tend to be the first to know.” (PHPT2)

Securing and maintaining funding is also something that is of importance to those involved in managing a Zooniverse project, and highlights the issue of sustainability in online citizen science.

“One of the biggest problems on my desk is making sure that all these [Zooniverse] projects exist in ten years-time when citizen science is not sexy and almost all our grants have come to an end.” (PHPT3)

246

However, it appears that the main obligation perceived by the scientific team is to ensure that something is done with the results generated by citizen scientists. Most of the scientists emphasised the importance of not wasting anybody’s time.

“People understand that we’re doing something with what they’re contributing as well.

We can show them we’re doing something with the data.” (PHPT2)

“…we’ve been entrusted with 18 million classifications so the main thing is we have to get as much science out of that as possible. I feel that the worst thing would be to waste people’s time. For Planet Hunters I feel very responsible that this happens.” (PHPT3)

The project scientists also articulated a responsibility towards the community of Planet

Hunters, and that they need to be present and available to answer questions, or disseminate information about the projects and the results. However, not all of the scientists get involved with interacting with the community to the same degree, and there are varying degrees to which scientists become embedded in the participant community.

“I think engage with the community is the responsibility of the science team, but I don’t think everyone in the science team has to do that.” (PHPT3)

One of the scientists I interviewed is heavily embedded in the Planet Hunters community and is highly visible both on the online discussion forum, and writes most of the entries on the project blog.

“I probably check Talk 4, 5, or 6 times a day….My goal when being on Talk is to make sure that information is being disseminated properly, and that when someone asks a question, we’re answering it the way it should be answered….we need to be around, and trying to teach people what actual science is….sometimes you want to go the extra mile.” (PHPT2) 247

Taking the time to interact with Planet Hunters participants can be time-consuming, and many of those in Zooniverse scientific teams are early career researchers, so they are under “pressures to publish” and to produce results.

View of the volunteer’s motivation

All of the scientists interviewed had been sent a summary of the survey results, so we used some of the findings as a starting point for our discussion. One of the more interesting areas for the scientists was the exploration of the motivations of the Planet

Hunters volunteers.

The opportunity to get involved in authentic scientific research was considered to be important by the scientists (as well as participants), and was felt to be one of the defining characteristics of the Zooniverse projects.

“…it’s always about the experience and I think for me it’s the authenticity….People get that they’re data and actual graphs – but they understand the abstract concept of looking for the signature of another world.” (PHPT2)

“I think it matters in the long run that it’s real.” (PHPT3)

That participants like to contribute or to help out was highlighted.

“I think it’s the sense that ‘today I did this thing’. You see that in the way the users, certainly the advanced users, talk about the project. I think there’s a pride in it.” (PHPT3)

“I think a lot of people might do it because they want to contribute, and this is definitely a solid way that they can contribute, and we can’t do it any other way.” (PHPT1)

The scientists felt that the possibility of making a discovery was a key motivator for participants and an important draw of the project.

248

“We did worry if people would come and look at these graphs – are people going to come and look at light curves? They do – that’s because we have really good ‘bacon’, and our bacon is you can help find planets.” (PHPT2)

The educational component of participation was raised by the community support manager, who has plans to increase the educational content of the Planet Hunters website.

“So one of the things I’m really looking forward to is attempting to educate our users in

Planet Hunters. Please come to the site and do the classification and also learn about exoplanetary physics while you’re doing it. From a base level up.” (PHPT4)

The fact that the hunt for exoplanets is a new and exciting field was highlighted by one of the scientists, and the number of new exoplanet discoveries in the last 10 years has been significant72.

Varying levels of involvement

There is an acknowledgement among the Planet Hunters project team that participants demonstrate varying degrees of engagement with the project. Some limit their involvement to classifying light curves when they have the time or inclination, and there are those who want to go further into the analysis of the raw Kepler data. The existence of a small community of core participants was discussed and some of the scientists had worked directly with these individuals.

“I have contact fairly regularly, at least a couple of times a week, with some of the users by email. So it’s discoveries they’ve made. I guess they coined the term ‘superusers’,

72 The official webpage of the Kepler Mission which lists the several thousand exoplanets discovered since the launch of the mission in 2009: http://kepler.nasa.gov/Mission/discoveries/. 249 they’re the volunteers who monitor the site and collect things and whatnot. They’re really keeping the project going!” (PHPT1)

“I think we have a group who have evolved out of the project interface who are just doing their own analysis…and I think that’s quite interesting. They talk to each other behind the scenes…I think there might be about 20….but they kind of have their own little hierarchical structure as well…although we still have a bit of authority over it…” (PHPT2)

The importance of this group is acknowledged, however, while the project team want to support this group and offer the opportunity for greater involvement to other participants, it was also recognised that this is not what all volunteers want from their participation in Planet Hunters.

“We talk a bit about a career in citizen science and that people should go from just clicking on stuff to writing papers and how they can reconnect to a professional. The problem with this type of rhetoric is that you end up sounding like you want everyone to go down that path, whereas I think a lot of our volunteers just want to have a hobby that takes a few minutes…and you can kind of get into trouble….” (PHPT3).

I spoke to one of the scientists about the pattern of participation in Planet Hunters where most participants contribute very little, and a small number of participants do the bulk of the work. We discussed the graphic that had been produced illustrating the pattern of participation for Planet Hunters (Figure 4.19, Chapter Four). The scientists stated that this is a characteristic pattern of participation in many of the Zooniverse projects.

“We have to cater to those who are among the little squares, and those who are among the big squares.” (PHPT3)

250

One of the key challenges for the project team is to manage the expectations and requirements of these different levels of participation.

Working with the participant community

While the value of online citizen science as a scientific research tool was made clear by the project team, a number of challenges associated with working with a community numbering in the thousands were also outlined. One of the issues highlighted was that it is time consuming to spend time interacting and communicating with the community of volunteers. Project scientists have full-time jobs doing scientific research, sometimes lecturing, and always involving the writing and publication of research papers.

“If I were to ask scientists to go on to Talk once a week and find five things to say, most would say they’re too busy….I’ve always been reluctant to mandate ‘you must spend three hours on talk’ or what-have-you.” (PHPT3)

The need to address other tasks and commitments has to be balanced with their obligations to the participant community, which suggests that these scientists draw a line between, or compartmentalise, these different obligations. It was evident that some of the scientists enjoyed this interaction more than others. One of the scientists did not interact with participants at all on the website, while another appeared to be highly committed to the community, and spoke about sharing the experience of scientific research.

“It’s about sharing the news when you have it and about sharing the process as you go along.” (PHPT2)

The same scientist spoke of project participants as ‘collaborators’, and the fact that this was their role demanded that time and effort be invested in communicating with them.

251

“I think you need to remember that you are dealing with people. These are not machines, and one of the things you have to recognise is you have to deal with that, and acknowledge and treat them as collaborators. I think it’s very important to bring them along on the ride.” (PHPT2)

They went on to add, that once you start to view participants as collaborators that “things get easier”. Another scientist spoke about trying to make everyone happy during earlier experiences in other Zooniverse projects, and speaking personally to participants who may have been dissatisfied in some way to make sure everything was alright. However, such an approach could not be maintained, and this individual stated that they didn’t actually have a responsibility to make everyone happy.

“I find the problems of running a citizen science project….[doesn’t finish sentence]. People are more complicated than galaxies!” (PHPT3)

In recent months there appear to be some changes taking place on the project regarding how the Planet Hunters project team communicate with participants, for example,

Facebook and Twitter are being used more widely to communicate and to disseminate results. I was able to discuss some of these changes with new community support manager, who confirmed that social media was going to be an area of focus not only for

Planet Hunters but for other Zooniverse projects. They clearly felt that a greater use of social media would improve and facilitate communication between project participants and the project teams and ultimately make the community ‘happier’. When asked how they would know if the community was happy, they stated that there would be more classifications.

252

“Productivity and happiness heavily correlate I think. I don’t think people would be there otherwise.” (PHPT4)

How ‘happiness’ would be gauged in the Planet Hunters community is unclear, but some economics-based research into happiness and productivity, has shown that individuals can be more productive, and produce better quality of work if they are happy (Zelenski et al., 2008, Oswald et al., 2009). Another point that arose during my discussions with project scientists that may also influence the relationship between the project team and the community is the fact that citizen scientists may have a different idea of what constitutes scientific research to those of the professional scientists. One scientist referred to the notion of “Hollywood science” where people had an unrealistic view of what actually constitutes scientific research, and perhaps as a result, had unrealistic expectations regarding the project, or of the project scientists.

7.3 Summary of results

The feedback from the survey illustrates that this group of respondents is predominantly male, well-educated, with a background interest in science, and in astronomy more specifically (RQ1 – Who participates in online citizen science?). Most (75%) contribute quite lightly in that they do not spend much time each week classifying light curves, and half of those surveyed (58) do not interact at all with other participants. This feedback helps to address RQ 5 (How can contribution to online citizen science projects be characterised?) and illustrates that contribution to Planet Hunters can be casual occurring when respondents have some free time. Participation can be solitary, and an individual can participate without interacting with others. This pattern of participation has been observed in other Zooniverse astronomy projects (Ponciano et al., 2014, Tinati et al.,

2014). Most (92) of the respondents felt they were making a contribution to scientific

253 research although many felt that rather than being at the forefront of scientific research, they provided a supporting role (RQ6 How do participants perceive their role in the project?).

Respondents are motivated to try Planet Hunters because they have a background interest in the science and because they want to make a contribution to an authentic research project. This was found to be an important motivator for participants in Galaxy

Zoo, another Zooniverse project (Raddick et al., 2010, 2013). The possibility that they may discover a planet also draws in many of the respondents. Most of these individuals continue to participate in order to carry on making this contribution to science, and hope that they will one day make a discovery themselves. The fact that the project task is relatively easy and the interface is accessible, are important in maintaining participation for a significant group of respondents. This feedback has helped to address RQ 2 (What motivations initiate and sustain participation in online citizen science projects?).

The interviews with participants have highlighted the importance of motivations that were identified by the survey. They have also demonstrated that taking part in Planet

Hunters may provide an opportunity for participants to reconnect with science, particularly for those who may have been formally educated in a science subject and who may have wanted to follow a career in science, thus identifying another important motivation for this group of citizen scientists (RQ 2). A small amount of dissatisfaction with the some features of the website is evident, and some interviewees display a lack of confidence in the accuracy and validity of their contribution. This may affect the motivation of some participants to continue their contribution to the project. Data from both the survey and interviews with citizen scientist volunteers suggests that participants do not generally interact with the project scientists unless they are forum moderators, or

254 have been involved in one of the exoplanet discoveries (RQ 4 How and why do project participants interact online?).

Interviews with the project team suggest that Planet Hunters is seen as an important tool for carrying out research, with the secondary benefit of providing a vehicle for communication with the community of participants (RQ 2). Most of this communication is one-way from the scientists to the citizen scientists, but there is some evidence that a small group of participants actively engages one of the scientists with their observations and wider interests relating to the Kepler data (RQ 4). There is an acknowledgement that some participants want to go beyond ‘clickwork’, but that the needs of more casual participants have to be considered alongside those who want to go further in their analysis. Managing a large community of volunteers can be difficult, and communication activities must be fit into a busy professional schedule of conducting research.

Planet Hunters involves a relatively straightforward and repetitive task than can be carried out with very little training or input from other participants (unlike Foldit). It provides a good example of ‘microvolunteerism’ in that interested individuals can participate whenever they have some spare time, and for a relatively short duration

(Paulos et al., 2011). Those with a greater interest in the science, and in examining the

Kepler data will be able to interact with other interested participants on the discussion boards and on Talk, and will also be able to interact directly with one of the project scientists, who are keen to bring on board enthusiastic citizen scientists, and will acknowledge their efforts in any resulting discoveries and publications (Schwamb, 2014).

255

Chapter 8: Discussion and comparison of results This chapter will integrate and discuss the findings of the three case studies, introducing a comparative analysis based on data produced by the different streams of evidence. I will explore explanations for the findings and consider them within the context of previous research (Holohan and Garg, 2005, Raddick et al., 2010, Krebs, 2010, Nov et al., 2011,

Rotman et al., 2012), and theoretical models including motivational frameworks (Clary et al., 1998, Batson et al., Ryan and Deci, 2000 &2009), the ‘reader-to-leader’ framework

(Preece and Schneiderman, 2009), and the ‘lightweight-heavyweight’ model of peer production (Haythornthwaite, 2009). Each research question will be considered in turn, with the emphasis on exploring evidence across the three case studies.

8.1 Who takes part in online citizen science projects? (RQ1)

This question was addressed by data obtained from the online surveys of (citizen scientist) participants, and through my observations as a participant. One of the earliest and most important observations was that despite the fact that each project has many thousands of registered participants, only a small percentage of this population actively contribute. This was particularly apparent in Foldit and Planet Hunters, where my analysis documents approximately 300 active contributors from a pool of over 500 000 registered participants and 160 000 respectively. In Folding@home, a group of highly active contributors from the community of overclockers and hardware enthusiasts was identified. This group make a substantial contribution to the project through the modification of their computers and their technical knowledge (see Chapter Four, Section

4.2.3). This research has ultimately focussed on these groups of more active participants because of their importance to these projects, and because they are more accessible to the researcher though project forums, discussions and blogs.

256

The analysis of the data on active participation demonstrates that what constitutes an

‘active’ participant varies from project to project and depends upon the project task and the level of difficulty involved (Section 4.4). Some tasks are more demanding than others, and may require more regular and sustained participation, not only to complete the project task, but in order to maintain a degree of familiarity and competency with the project tools, particularly if they evolve over time such as in Foldit where participants have become a self-organising community, taking on specific roles. In distributed computing projects, such as Folding@home, pre-existing communities of overclockers constitute a significant proportion of the group of active participants (Bohannon, 2005), illustrating how the Internet can facilitate connections across networks.

Neither the discrepancy between registered and active participants, or a consideration of what constitutes an active participant in a project, has been discussed in detail in previous research (Holohan and Garg, 2005, Krebs, 2010, Nov et al., 2011, Raddick et al, 2010,

2013). Furthermore, communication efforts by some of the scientists involved in these projects, as well as a number of news and magazine articles, refer to and emphasise the numbers of registered participants (Bonetta, 2009, McGonigal, 2010, Nielsen, 2012,

Wiederhold, 2011, Savage, 2012, Cossins, 2013, Schwamb, 2014c). While these articles and publications help to generate interest in a project, they can also give the impression that there are more people contributing than is actually the case, and create an impression that many thousands are making important contributions to these projects, when in fact, the numbers are more modest. The observation that only a small proportion of those showing an interest in a project will go on to make an active and productive contribution, has been made in relation to other types of online projects and endeavours (Kittur, 2007, Preece and Schneiderman, 2009, Schroer and Hertel, 2009).

257

This pattern of contribution will be explored in detail in Section 8.5 in relation to RQ5

(How can contribution to online citizen science projects be characterised?).

The online surveys have provided more information about the active participants in these projects, and include some demographic information that has not been explored in previous research, such as formal qualifications in STEM subjects, and participation in other science-related activities. I have also reported demographic information relating to participants in an online citizen science game. When considered together, the results from the three online surveys illustrate that these respondents have a number of demographic features in common.

 They are predominantly male (there were 45 female respondents from a total of

562, 8%).

 Respondents are mainly from developed countries (549 respondents, 98%).

 Most of the respondents are well educated with the majority having at least an

undergraduate degree (335 respondents, 60%). A high proportion of those who

are graduates have qualified in a STEM subject (81% of graduates, 272

respondents). Ninety-seven respondents are currently students.

 Almost one third of respondents work in IT-related professions (177 respondents,

31.5%).

 Most of the respondents are engaged with science in some way. Practically all

(with the exception of a few) had taken part in other science-related activities in

the previous year, and just over half (293, 52%) had taken part in other citizen

science projects.

258

8.1.1. The predominance of male respondents

Much of the previous work that has measured demographic characteristics of participants in online citizen science projects has found an overrepresentation of men (Estrada et al.,

2013, Krebs, 2010, Raddick et al., 2013, World Community Grid, 2013). My survey data also found that the overwhelming majority of respondents to the surveys were male (this was especially true in the Folding@home group where more than 98% of the respondents who responded to the survey were male). There are a number of possible explanations for this pattern of participation.

Some research has shown that men are more likely to engage with science than women and are more likely to take an interest in new technology and scientific developments

(von Roten, 2004, RCUK, 2008). There may also be a disparity in what type of science appeals more to women than men, and some areas of science (particularly the physical sciences) attract less women than men professionally (Eurobarometer, 2010, Ivie and

Tesfaye, 2012). For example, astronomy is a subject that has been male dominated both in the professional and amateur arena (McWhinnie, 2011, Bowdley, 2009). Previous research has also shown that public engagement events in astronomy also attract more men than women (Entradas et al., Curtis, 2013). It is perhaps unsurprising therefore, to find more male participants in an astronomy-based citizen science project such as Planet

Hunters.

However, both Foldit and Folding@home deal with research in biological science, which tends to have a better representation of women both in formal education and professionally (albeit at relatively junior levels compared with men) (Bowden, 2012,

Howard Eckland, 2013). A recent UK study also showed that women tended to be interested in areas of science that were related to health and medicine (Ipsos-MORI,

259

2011). Both Foldit and Folding@home address the underlying mechanisms associated with some serious diseases such as Alzheimer’s, cancer and influenza, so the low proportion of women among the survey respondents was surprising.

Some previous research has found that gender has an effect on internet use, and several studies have found that men tend to spend more time than women on the Internet (Jones et al., 2009, Helsper, 2010, Joiner et al., 2012). Not only do men tend to spend more time on the internet, but they are also more likely to use it for ‘entertainment’ purposes and to play games (Jones, et al. 2009, Helsper, 2010). These observations could also help to explain the higher proportion of men in the projects explored in this research, and increase the likelihood that men will learn about these projects via online articles or searches.

However, the format of these projects is of importance, and the fact that Foldit is a game and Folding@home is a distributed computing project could influence their appeal to women. While the number of women who play computer games has grown rapidly over the past decade, they tend to play different types of games than men (Internet

Advertising Bureau, 2011). For example, they do not play as many online multi-player games and they tend to play more games on mobile devices (Green, 2012). One study also found that due to less leisure time available, fewer women played computer games, and those that did, played them for much shorter bursts of time (Winn and Heeter, 2009).

These effects of gender on game-playing habits could help to explain the smaller number of female Foldit players (Entertainment Software Association, 2012, Green, 2012).

In distributed computing projects, all previously published research has shown that the majority of participants are male, and this includes studies with very large sample sizes

(SETI@home, 2006, World Community Grid, 2013, Estrada et al, 2013). The appeal of 260 distributed computing projects to hardware enthusiasts and overclockers, and the lack of women among this community in the online survey (as well as on the overclocker forums that were observed), may explain the extremely low proportion of women in the

Folding@home sample. Folding@home does not appear to be as widely promoted by the project team as Foldit and some of the Zooniverse projects, and most of the survey respondents report hearing about the project through websites and forums relating to computer hardware and overclocking. As a result, a smaller proportion of women may be hearing about the project. However, it has been harder to reach more ‘passive’ participants in Folding@home (those who just download and run the programme) and an investigation into these participants could possibly reveal greater female representation.

To date, only a limited number of online citizen science projects have been explored (see

Chapter Two, Tables 2.1 and 2.2), and the trend toward greater male participation may not be a feature of other projects. The results of this research could have implications for the wider accessibility and appeal of online citizen science projects, particularly distributed computing ones. Those setting up an online citizen science project may want to consider whether they want to appeal to a wider base of participants. The scientific discipline of the project, the project format, how and where project managers communicate about the project may have an influence on the gender profile of their participants.

8.1.2 Level of education and general interest in science

Another notable feature of these three groups of respondents was how well educated they were and 60 % (335) had a university education. Of the remaining 227 respondents,

97 were currently studying. Two other studies on Zooniverse participants have also shown a high representation of university graduates among study respondents (Reed et

261 al., 2013, Raddick et al., 2013). Previous research on attitudes towards science has shown that those with a greater level of education are more interested and engaged with science (Ipsos-MORI, 2011, RCUK, 2008). Such individuals may consume more science- related content and come into contact with sites and publications which promote and discuss online citizen science projects.

Many respondents (272, 48%) had formal qualifications in STEM subjects although there were very few respondents (36 in total) who were employed as scientists or were in medicine. These individuals bring some expertise to the projects, either through their knowledge of the related science (e.g. two of the Planet Hunters interviewees had degrees in astronomy), or knowledge of the ‘scientific process’ (e.g. the importance of rigorous data analysis, publication of results, collaboration etc.). Feedback from some of those who took part in the Planet Hunters interviews highlighted a desire to be

(re)connected with science, and that their involvement in the project offered them to the opportunity to get involved in an area that was once of great interest or importance to them. Those with a formal qualification in a STEM subject have already demonstrated an interest in science and technology, so online citizen science games may appeal to these interests.

The tasks associated with Planet Hunters and Folding@home were not especially complex, and there is no requirement for any previous knowledge or experience of scientific research in any of the projects investigated. However, Foldit is a difficult game to learn and play, and requires complex problem-solving skills and spatial awareness. Of the three projects, Foldit respondents had a greater percentage of graduates (65%), and a quarter of respondents were educated to masters or PhD level. The level of difficulty associated with the project task compared with the other two may explain this finding.

262

Practically all of the respondents demonstrate a wider interest in science and report taking part in science-related activities and reading scientific publications and online content (only three respondents reported not taking part in any science-based activities in the previous year). More than half (52%, 293 respondents) had taken part in other citizen science projects. This high level of engagement with science may be indicative of the appeal of online citizen projects to those who have been described previously as

‘confident engagers’ (Ipsos-MORI, 2011) or ‘fans of science’ (Priest, 2009). According to this research, such individuals are usually better educated, more affluent than other sections of the population, and have more positive attitudes towards science and scientific developments. In a recent UK study, 14% of the population fit into this classification (Ipsos-MORI, 2011).

Among survey respondents, there were a high proportion of those in IT-related professions (177, 31.5%). This could be related to the presence of a large number of hardware enthusiasts and overclockers in the Folding@home sample. However, the other two projects also had significant numbers of IT professionals. This suggests that online citizen science projects may be more appealing to those who are confident in using computers, or who are more technically proficient. Such individuals bring skills which have been beneficial to these projects. This may be seen in the advent of ‘recipes’ in

Foldit and the coding of moves with the Lua coding language, and in the contribution of the Beta Team in Folding@home.

The most recent Oxford Internet Survey (2013) refers to confident internet users as ‘e- mersives’, and defines them as individuals who are comfortable and naturally at home in the online world and use the Internet for entertainment purposes, to meet people and to make their lives easier (Dutton and Blank, 2013). Based on survey and interview

263 feedback, as well as my observations, it is likely that this term accurately describes active participants in these three projects.

8.1.3 The appeal of online citizen science projects

The projects that have been investigated as part of this research appear to appeal to male, well-educated and scientifically engaged individuals who are also likely to be confident with computer technology. The findings of this research appear to corroborate the findings of other studies (Holohan and Garg, 2005, Krebs, 2010, Nov et al., 2011,

World Community Grid, 2013, Raddick et al., 2013), yet there remains a small sample of projects that have been investigated in any detail, and most of these have been distributed computing projects and astronomy-based distributed thinking projects (see

Chapter Two, Tables 2.1 and 2.2).

The lack of diversity in participants in some online citizen science projects may be related to a phenomenon known as ‘threshold fear’ (Gurian, 2005). This has been examined in relation to attendance at museums, art galleries and other public cultural institutions. It has been defined as the constraints people feel that prevent them from participating in activities that are targeted at them (Gurian, 2005, Simon, 2012). In the case of physical spaces there may be tangible impediments that prevent some people from attending or participating (such as location, or cost of entry), but there are also important socio- cultural factors such as gender, ethnicity, age, class background and personal history that influence who takes part in these activities, and what kind of experience they have if they do participate (Dawson and Jensen, 2011).

In the case of online citizen science projects, these socio-cultural factors may also be of importance and some individuals may think that these activities simply aren’t meant for them. The scientists who are involved in Planet Hunters (and other Zooniverse projects) 264 have tried to ensure that the project task is relatively straightforward and have provided tutorials and other educational material. Tutorial training (in the form of intro puzzles) has also been provided by the Foldit project team. However, this may not be enough to increase the appeal of the projects to those outside of the ‘confident engagers’ group. In addition to where and how projects are promoted, there may also be design features that influence their wider appeal e.g. the overall appearance of the website and ease of navigation, and the presence of online forums or internet relay chat. Whether this is the case merits further research.

One reason for the relatively small numbers of active participants observed may be unrelated to ‘threshold fear’, and may instead be related to something that has been referred to as ‘participation bandwidth’ (McGonigal, 2008). This refers to the total amount of time we have available for online activities. According to McGonigal, there are ever increasing numbers of social networks to join, new wikis to edit, new content to contribute and new games to play. We are exposed to more opportunities to contribute than we could possibly accept, and we only have so much time to contribute to online ventures. Therefore, online citizen science projects have to compete with other networks and online interests for the attention of those who are active online, particularly those who are ‘e-mersives’.

The results of this research suggest that these three online citizen science projects have created opportunities for small groups of ‘confident engagers’ to become involved in authentic scientific research. These distributed volunteers have responded to a range of scientific challenges, and have self-organised into various project roles and teams to produce new knowledge. These findings also highlight the importance of making a

265 distinction between ‘registered’ and ‘active’ participants, and that what constitutes an

‘active’ participant, will vary between projects.

8.2 What motivations initiate and sustain participation in online citizen science projects? (RQ2)

Table 8.1 presents a summary of the results obtained through the online surveys, and lists the most important motivations that initiate and sustain participation for citizen scientist volunteers in each of the three projects. The number of individuals who gave these responses is in brackets. In most cases, two or three motivations are predominant.

Table 8.1 Summary of main motivations that initiate and sustain participation for citizen scientists

Foldit (n=37) Folding@home (n=407) Planet Hunters (n=118)

Motivations 1. Contributing to research 1. Making a contribution (to 1. Interest in the science that initiate (22). research or a worthy (56). participation 2. Interest in science (13). cause)(207). 2. Contributing to research 3. Intellectual challenge 2. Fully utilise computing (40). (10). power / hardware 3. Chance to make a 4. Curiosity (8). enthusiast (128). discovery (26). 3. Personal experience of 4. Goals of project diseases being researched important (11). by project (74). 4. Admire goals of the project (68). Motivations 1. Contributing to research 1. Making a contribution 1. Contributing to research that sustain (14). (150). (39). participation 2. Interaction with other 2. Ease of use / accessibility 2. Chance to make a players / community (13). (75). discovery (29). 3. Developing skills / 3. Overall idea / ‘concept’ of 3. Easy to take part (21). learning (9). project (54). 4. Community (10). 4. Opportunity to be 4. Competition (47). creative (8).

8.2.1 Initiating participation

For all three of the projects making a contribution is one of the most important motivations for participation. This was mainly making a contribution to scientific 266 research, although Folding@home participants tended to make a differentiation between making a contribution to science and making a contribution to a ‘worthy’ cause.

A background interest in science was also an important motivator for Foldit and Planet

Hunter respondents. An interest in the science and the desire to make a contribution were also important in the two other distributed thinking projects that have been the subject of previous research (Nov et al., 2011, Raddick et al., 2010). However, a background interest in the science was not mentioned by as many Folding@home respondents, although in this group (given the involvement of overclockers) a background interest in computing can be considered a related motivator.

A few motivations initiating participation varied between the projects. For example, the intellectual challenge of the game was important for Foldit respondents, as was their curiosity about the project (perhaps relating to the re-packaging of a scientific research problem into a game format). In Planet Hunters, the opportunity to discover an exoplanet was important for over a fifth of respondents. Although this is how the project is ‘marketed’, and its tagline on the Zooniverse website is ‘Find planets around stars’. In

Folding@home, the involvement of overclockers meant that the opportunity to fully utilise computer processing power was an important motivator for a large proportion of these respondents. This was also found to be an important motivator for participants in the World Community Grid (2013).

As Folding@home is concerned with research into the underlying causes of many significant diseases, a significant proportion of respondents stated a personal experience of a disease such as Alzheimer’s, Parkinson’s or cancer (either themselves, a friend or family member) as a reason for joining the project. While Foldit is also concerned with how protein folding may relate to disease, this was not given as a reason for participation 267 by many of the respondents to the Foldit survey suggesting that its format as a game may be more important when it comes to attracting new participants.

Feedback from the interviews with participants (citizen scientists) in Foldit and Planet

Hunters mirrored the results of the online surveys, and participants re-stated their desire to make a contribution to scientific research, and an interest in the background science.

The latter was especially true for the Planet Hunter interviewees, many of whom took part in amateur astronomy activities.

In Folding@home I spoke specifically to individuals who were involved in the Beta Testers and in overclocking. Those in the former group appeared to be motivated to join so that they can share their knowledge and expertise in order to make the project more successful. Those in the overclocking group wanted to make a contribution to the project, but also wanted to develop their technical knowledge relating to computer hardware. A small number also mentioned the competitive aspect of their involvement in

Folding@home.

The results from the surveys and interviews suggest that both the research goals of an online citizen science project and the subject area are important considerations for potential participants.

A project may need to demonstrate that there is a relevant and useful research problem to be investigated and that participants can actually make a difference. Members of the

Zooniverse team who I interviewed also stressed the need to effectively utilise the efforts of citizen scientists and not waste anybody’s time (see Chapter seven, Section 7.2.2). This utility of citizen scientists’ contributions may be demonstrated by promoting the discoveries that ‘ordinary’ citizens can make. For example, this has been done

268 successfully in some of the Zooniverse projects, where participants have made a number of serendipitous discoveries (Cardamone et al., 2009, Lintott et al., 2009), and in Foldit, where Foldit teams have been co-authors on important scientific papers (Khatib et al.,

2011a, Khatib et al., 2011b). Folding@home has a link to all of the papers resulting from the project (including a summary of the findings for non-specialists) and lets participants know via the project blog when new publications appear. Making a contribution to good quality science was also alluded to in some of the follow-up interviews with

Folding@home participants. That potential participants can make an important contribution appears to be key in the way some of these projects are pitched and communicated about both by those managing the projects, and also by those writing about them in magazine and news articles, both print and online (Bohannon, 2009,

Bonetta, 2009, Borrell, 2013, Hand, 2010).

Individuals are more likely to participate in a project that is also in an area of science that is of interest or relevance to them. While this was stated explicitly by a number of Foldit and Planet Hunter respondents in the surveys, interviews, and within discussion threads I observed though my own participation, it was more implicitly stated by Folding@home participants. The relevance of this project was illustrated in references by participants to loved ones who were suffering from the various diseases that are being investigated by the scientists associated with Folding@home. I also observed this strong motivation while exploring several of the overclocker communities, and many of these communities have discussion threads which have become virtual memorials to friends and family who have been affected (see Chapter Four, Figure 4.15). These individuals are motivated by the potential applications of this research and the relevance to their own lives. Previous research in citizen science, particularly work exploring communities and local

269 environmental issues, has also highlighted the importance that personal relevance plays in motivating participation (Irwin, 1995).

8.2.2 Sustaining participation

For all three projects the opportunity to make a contribution to science is also one of the main motivations that sustains participation. Again, as in initiating participation, it may be that the utility of citizen scientists’ contributions need to be made clear to participants in some way if they are to continue devoting their time to a project. All three of the projects have made an effort to do this through the material they provide on the project website and in how the projects have been communicated and promoted. The value of citizen scientists’ contribution is also demonstrated in some instances through the acknowledgement of contribution on publications (Lintott et al., 2009, Khatib et al., 2011,

Schwamb et al., 2013, Schmitt et al., 2014). While there appears to be a similar ‘hook’ of making a contribution for getting people to begin participating, the goals of the project, and the project task may mean that the subsequent experience for participants is very different from one project to the next. These differences will be explored in Section 8.3, when RQ 3 (How do motivations vary between different types of online citizen science projects and their related tasks?) is considered.

8.2.3 Models of motivation

All of the reasons for taking part in the three projects, as well as what participants liked best about participating, were listed. These were then considered in light of the motivational frameworks that have been utilised by other researchers looking at participation in online citizen science. While no single framework could be applied to all of the observed motivations, the work of Ryan and Deci (2000, 2009) on intrinsic and extrinsic motivations was the most relevant to my findings, and many of the motivations

270 given by citizen scientists for their participation could be classified in this way. According to Ryan and Deci (2000), intrinsic motivation exists when an activity is enjoyable, or when it promotes feelings of fulfilment and competence. There is no need for an external reward because the activity is inherently interesting and fulfils some of our basic psychological needs such as feeling capable or useful.

Extrinsic motivation is in operation whenever an activity is done in order to attain some separable outcome. However, extrinsic motivation is not merely a question of punishment or reward. It exhibits a range of expression that is related to the degree of autonomy experienced by an individual. This is also known as ‘self-determination theory’, which maintains that although an activity may not be interesting or appealing, it is personally endorsed in some way and the individual has a feeling of choice (Ryan & Deci

2009). This is in contrast to compliance, when an individual carries out an activity because of an external control (e.g. avoiding punishment). Motivations based on compliance are not relevant when it comes to participation in online citizen science, however, extrinsic motivation based on introjection, which is defined by Ryan and Deci

(2009) as ego involvement or the desire for approval from others, and identification, where an individual has identified the value of the activity, do appear to be relevant. In the latter case, an individual may continue an activity they find dull or boring if they believe it may ultimately be of relevance or importance (this is evident in survey feedback from Planet Hunters participants, several of whom described the task as monotonous).

While the approach of Ryan and Deci (2002, 2009) was relevant for many of the motivations for participation articulated by citizen scientists, some were not entirely explained by this framework. For example, one of the most commonly cited reasons for participation, the desire to help and to make a contribution to scientific or medical

271 research, is based on an altruistic motivation or empathy that has more in common with other types of more general ‘community-based’ voluntary behaviour as detailed by Clary et al (1998) and Batson et al. (2002). An altruistic desire to contribute to the ‘public good’ has also been observed among participants of other types of commons-based peer production such as writing open source software or Wikipedia articles (Kuznetsov, 2006,

Nov, 2007, Oreg and Nov, 2008). In addition to altruism, another important internal motivation is the desire to work with and be a part of a community; to cooperate and collaborate, in this instance as part of distributed social group. This motive has been identified in previous work on those who write open source software, and has been linked to a basic human need for belonging (Hars & Shaosong, 2002, Maslow, 1943).

Previous work on open-source software has also highlighted an important external motivator that relates to ‘expected future returns’ (Hars and Shaosong, 2002). Within the context of open-source software, this means that an individual may be rewarded for their involvement sometime in the future in the form of revenues from related products and services, or career advancement through marketing and showcasing their technical skills

(Hars and Shaosong, 2002, Hertel et al., 2003, Oreg and Nov, 2008). However, none of these are guaranteed. There appears to be a parallel to this motivation among some participants of the online citizen science investigated here, particularly those who have a more personal stake in the outcome of the research. For example, many respondents to the Folding@Home survey stated that their involvement in the project was the direct result of a loved one (or they themselves) being affected by one of the diseases being researched by the Folding@Home scientists. Some of the respondents to the Foldit survey also expressed this sentiment. Many of these individuals also hoped that their involvement would result in the development of a cure or therapies for these conditions.

272

Again, there is no guarantee that any of these participants will see these things in the near future, yet this remains an important motivator. Such a motivation could also be considered ‘enlightened self-interest’.

A framework, or classification system, has been developed that is based largely on the work of Ryan and Deci (2000, 2009) but also takes into consideration some of the motivations that have been identified in previous work on community volunteering, and participation in open source software. This framework (Table 8.2) seeks to incorporate all of the major motivations that have been articulated by respondents to the three online surveys, and build them into a model that represents a hierarchy of motivations, from high to low granularity.

At the highest level (Level 4), motivations can be classified as either internal factors, which are rooted within the individual, or they can be classified as factors that are external to an individual (Hars & Shaosong, 2002). At Level 3, internal factors can be subdivided into intrinsic factors, altruism and community, while external factors can be divided into extrinsic factors and expected future returns. These can be further subdivided (Level 2) into a number of elements that have been identified by Ryan and Deci

(2000, 2009) as the components of intrinsic and extrinsic motivation. A further breakdown of altruism, community involvement and expected future returns, and their relevance to online citizen science projects has also been illustrated at this level. Level one represents the lowest ‘granularity’ of motivation. Respondents articulated motivations that were either Level One or Level Two

273

Table 8.2 Motivational framework based on feedback to online surveys by Foldit,

Folding@home and Planet Hunters participants

Level 4 Level 3 Level 2 Level 1 task granularity

Internal Factors Intrinsic motivations Enjoyment Relaxing Visual appeal Fun Fulfilment Background interest in science Participation in authentic research Allows creativity Learning opportunity

Competence Intellectual challenge Using skills Formal qualifications not required Different ways to contribute

Altruism Making a Contributing to scientific research contribution Contributing to a worthy cause Helping scientists Community Interaction with Work with others toward common others goal Make friends External factors Extrinsic motivations ‘Ego enhancement’ Points (introjection) Rank Making a discovery wider recognition Positive feedback from scientists

Identification Goals of the project are important

Expected future Medical / scientific Research publications returns breakthroughs New drug therapies Cures

Most participants in this research have expressed more than one reason or motivation for taking part in their project and they can incorporate both internal and external factors.

This research has also shown that their motivations can change over time. However, internal motivations appear to be dominant in the case of online citizen science, as making a contribution (altruism), and an interest in the science (the intrinsic motivation of fulfilment) are the two most commonly cited reasons for participation. ‘Ego enhancement’ rewards in the form of points, rank and reputation appear to be of greater 274 importance to the community of overclockers who participate in F@H, and for some

Planet Hunters participants who want the recognition associated with making a discovery.

Whether this model is applicable to other online citizen science projects is of interest, and it may be of relevance to scientists and developers considering setting up an online citizen science project. Results from previous studies, particularly work on other Zooniverse projects (Raddick et al., 2010, 2013) and some work on distributed computing projects

(Holohan and Garg, 2005) suggest that levels two through four may be applicable and able to describe the motivations reported by participants. However, this would require further investigation and discussion.

8.2.4 Motivations of project scientists and developers

The motivations of those involved in setting up and managing online citizen science projects has not been considered in previous research in any detail. The scientists interviewed who are involved in Planet Hunters and Folding@home set these projects up in order to accomplish specific research goals based on the analysis of large volumes of data. In order to achieve their research aims in a realistic timeframe (years rather than decades) these scientists involve non-specialist volunteers. A precedent had already been set in the form of contributory, ecology-based programmes (Mayer, 2010, Dickinson et al., 2010, Roy et al., 2012) and the advance of digital technologies and the internet has provided the infrastructure that enables online citizen science projects. In the case of

Planet Hunters, a digital infrastructure was already in place in the form of the Zooniverse.

The developers of Foldit work with a team of biochemists to ensure that Foldit is guided by specific research aims and goals. The game was designed to take advantage of the diversity that exists in the problem solving abilities and game strategies of each individual player. The developers interviewed were motivated to get involved in Foldit because 275 they wanted to work with the wider community that were external to their research institution, and because they saw a potential for games in addressing specific scientific problems (Cooper, 2011).

In appears therefore, that the primary motivator for the project teams I interviewed is to tap into the pool of citizen scientist volunteers to get help with scientific research. Citizen scientists help professional researchers to carry out tasks that would be highly time- consuming and repetitive, thus freeing them up for more in-depth analysis that requires more specialist knowledge and skills. They also carry out tasks that (at this point in time) cannot be carried out by computer algorithms, and tasks that rely on distinctly human approaches to problem solving. In this sense, citizen scientists are providing a free labour resource to these groups of scientists.

However, scientists are also able to meet other professional objectives through their involvement in online citizen science projects. For example, the Planet Hunters project team see the project as a way to engage with their volunteers and in the words of one of the scientists “bring them along on the ride” - the ‘ride’ being an insider’s view of professional scientific research. The Zooniverse community manager spoke of developing informal educational programmes for Planet Hunters participants (something that

Zooniverse projects have not tended to do in the past), and trying to encourage participants to learn more about astrophysics.

While ‘engagement’ more often than not refers to educating volunteers, or telling them about the research process, some of the project scientists try and encourage them to become more involved in the research process and to start asking questions of their own.

A recent project in the Zooniverse called ‘Galaxy Zoo Quench’ has tried to involve participants more closely in the research process and citizen scientists are able to get 276 involved with different levels of data analysis and collaborate on the preparation of a journal article (Simpson, 2013). However, this project is overseen by a group of professional astronomers, and they (not the citizen scientists) have formulated the research questions.

Foldit scientists and developers also regularly engage with the players and hold regular online ‘chats’ that any participant can attend and pose questions to the project team (see

Chapter Four, Figure 4.7). Members of the Baker Lab (including Professor David Baker) have been involved in the production of blog posts, podcasts and short films about the project that are regularly featured on the Foldit website. External presentations made by scientists have also been shared with players. However, the Foldit developers interviewed stated that because there are so many participants to engage with there isn’t always the time (or the human resources) to do this enough justice. Over time, and perhaps as a result of this, the Foldit community has become a self-organising research community, with a small number of players occasionally working with the project team.

This sentiment was expressed by nearly all of the project scientists that were interviewed, and could also be seen as a reflection of how they view citizen science more generally.

For some of the project scientists, a conflict between interacting with citizen scientists volunteers and carrying out their ‘day jobs’ is evident. This suggests that for some of these scientists, citizen science may not be considered a core research activity. This issue, as well as some of the other issues surrounding ‘management’ of large groups of citizen scientists, as well as the varying degrees to which scientists become ‘embedded’ in the project community, will be discussed in greater detail in Section 8.4.2.

277

8.3 How do motivations vary between different types of online citizen science projects and their associated tasks? (RQ 3)

Motivations that initiate participation in online citizen science projects share a number of similarities between the three projects investigated, and most of the individuals surveyed become involved in order to make a contribution, and because they have a background interest in the science relating to the project. However, the nature of the project task and the goals of the project also influence other (less cited) motivations that initiate participation. For example, my research indicates that Foldit players are drawn to the game because of the intellectual challenge of the task, and because they are curious to see how a scientific problem has been re-packaged as a game. For Planet Hunters participants, the data indicate that they want the opportunity to make a discovery which they feel is important in some way. For many (74, 18%) Folding@home participants, they have some personal experience of the diseases that are being investigated by the Pande

Group, and they want to be able to help the scientists find a cure, or develop new drugs.

Where the projects show greater divergence, is in the factors that sustain motivation (see

Table 8.1). This has implications for the design of projects, and in the setting of project tasks.

While making a contribution sustains motivation in all of the projects, one of the key differences appears to be the importance placed on the interaction with other participants and the online community. For example, for Foldit players, the importance of interaction with other players was emphasised in both the online survey and in the follow-up interviews. My experience as a participant also highlighted the importance of the community and I observed a high level of interaction (see Chapter Four, Section

4.1.5). Players help each other learn to play Foldit, they collaborate and co-operate with

278 each other on the science puzzles, and many make friendships through their participation. This interaction also contributes to some of the other motivations that help to sustain participation such as the ability to develop skills, and to work creatively. This was especially apparent in the interviews where participants spoke of “theorycrafting” and collaboration, and their pride in the achievements of the community. The importance of this interaction is related to the difficulty of the project task, and through sharing different approaches and perspectives, better results may be achieved both individually and collectively.

This interaction was emphasised to a lesser extent through the interviews with the

Folding@home Beta Testers and members of the overclocking community. Some Beta

Testers members spoke of the enjoyment they derived in working with others to solve some of the software issues the project faces. Overclockers referred to the fact that members of this community, while competitive, were also happy to share expertise and help others with technical issues. One of the appeals of Folding@home for many of these participants was that they could combine this interest with making a contribution to scientific research.

Only a small percentage of Planet Hunters participants (10 respondents, 9%) spoke of a project community on the online survey, with half (9) stating in the interviews that they didn’t interact at all with other project members. The survey indicates that this is a solitary activity for a significant number of the respondents. The task on Planet Hunters is relatively straightforward and completed individually, although a participant may ‘flag’ an interesting item, or ask for advice about spotting a planetary transit on one of the discussion boards. In fact, the ease of participation keeps 21 respondents (18%) participating. Most of those who took part in Planet Hunters survey and interviews work

279 by themselves classifying the light curves, dipping in and out of the project when they have the time and inclination. It is this pattern of participation that is appealing for these individuals allowing them to ‘microvolunteer’ when time and inclination allows, while still enabling them to contribute to authentic scientific research (Paulos et al., 2011).

Thus, the importance of ‘the community’ in sustaining involvement in a project appears to be related to the complexity of the project task, and the community is more prominent in projects, such as Foldit, where the task is difficult, and where new participants may need the help of more established participants in order to learn how to play, and among the Folding@home Beta Testers who collaborate with project developers to address software issues.

The possibility of discovering a new planet remained an important motivating factor that sustained participation in Planet Hunters. The possibility of discovery was referred to by one respondent as the “thrill of the chase” and has also been noted as an important factor that retains participants in Galaxy Zoo. In Galaxy Zoo however, it is the possibility that the next image will be even more beautiful than the last (Raddick et al., 2010,

Sproull, 2011).

Another factor that was highlighted in the Planet Hunter interviews that may play a role in sustaining participation, was the opportunity to re-connect with science. This appeared to be important to those who had a formal qualification in science who were unable to pursue a career in science and for some who had been deeply interested in science during childhood. Planet Hunters may also offer a way back in to science more formally, and two of the Planet Hunters participants (as well as a few Foldit and

Folding@home participants) had enrolled in formal education courses to improve their knowledge of the science associated with the project. Involvement in an online citizen 280 science project may also offer the opportunity for a professional scientist to become involved in another discipline.

In Folding@home, motivations that sustained participation were related to the involvement of the overclockers and hardware enthusiasts. The fact that the project and project interface are easy to use was mentioned by 75 respondents, and in many cases this feedback was provided with a reference to overclocking activity. For other participants (47) the competitive element was important, and again, this aspect of participation in distributing computing projects appeals to hardware and overclocking enthusiasts.

Opportunities for informal learning were not explicitly referred to by many of those responding to the survey or taking part in the interviews, and this applies to all three of the projects investigated. When asked directly what they had learned by participating in their project during the interviews, the majority stated that they had learned about the related science to some degree. However, this was not an important factor in sustaining participation, although learning about game-related skills helped to sustain participation for approximately a quarter (nine) of the Foldit survey participants.

The results of this research show that the main factors that initiate motivation (wanting to make a contribution and an interest in the science) are similar between the three projects but that the less commonly cited reasons for participating are related to the project task, and to the overall goals of the project. Factors that sustain motivation show a greater divergence between projects, although making a contribution is still the most important motivation for remaining involved in all three projects, as are task-related motivations (such as discovering a new planet, or being able to combine participation with an interest in computer hardware or overclocking). The level of difficulty of the 281 project task influences the importance of the online community in sustaining participation. Thus, Foldit participants value this aspect of participation more than those participating in Planet Hunters and Folding@home, although the importance of a community is important to some Folding@home participants involved in overclocking or the Beta Testers. Scientists and developers considering setting up an online citizen science project need to consider the dynamic nature of motivations (Batson et al. 2002,

Rotman et al., 2012), and that they are influenced by the design of the project

(particularly any opportunities provided for interaction with other participants) and by the complexity of the project task.

8.4 How and why do project participants interact online? (RQ4)

Interaction in online citizen science projects can occur between citizen scientist participants, and between citizen scientists and members of the project team. This interaction can take place in a number of online locations such as forums, blogs, and internet relay chats. Project participants were asked about interaction with others on the online surveys and during the interviews. I was also able to provide a partial, ‘snapshot’ assessment of the degree and extent to which participants interact via experience as a participant-observer.

8.4.1 Interaction between citizen scientists

Of the three projects, Foldit players reported (via the survey) the greatest level of involvement in the online project community. Of this group of respondents, 84% (31 players) report regularly interacting with other players online. The ‘community’ appeared to be one of the most important features of Foldit, and players created and fulfilled a variety of project roles (see Chapter Four, Table 4.1) . Some Foldit players (both on the survey and during the interviews) spoke about the community with pride and warmth,

282 highlighting the social dimension of online participation. Players talked about the friendships they had made online and with fellow team mates. One player referred to his team as his “folding family”. General levels of interaction between Foldit players are also greatly enhanced by the presence of a synchronous ‘global’ chat window.

In Folding@home, approximately 80% of survey respondents stated that they read or post in forums relating to the project. Through the interviews, the sub-communities within the Folding@home sample, the overclockers and the Beta Testers, also report regular use of online forums to discuss issues relating to their involvement in the project.

Similarly to Foldit, comments were made by members of the overclocking community and the Beta Testers in Folding@home about making friends online, and about the enjoyment of working with others on technical problems.

In comparison, a smaller proportion of Planet Hunter respondents take part in online discussions, and most tend to work alone while carrying out the project task.

Approximately half report reading content on the online discussion boards or Talk function, but only 25 respondents (21%) have ever posted content. Data provided by one of the project scientists on the number of Talk comments posted on Planet Hunters suggests that nearly a third of the total number of comments made have been posted by only 48 different individuals (Raddick, 2013). Taking part in the Planet Hunters discussion boards does not have the immediacy of taking part in an internet relay chat conversation, or even taking part in one of the overclocker forums, which appear to have a relatively rapid response rate if one posts a comment. My experience in Planet Hunters was that responses to comments on the discussion board and on Talk can take a while, if indeed there are any responses at all. According to one of the Planet Hunters scientists, a small group of core participants works together and analyses the raw data from the NASA

283

Kepler project. She stated that these individuals privately email each other and one of the project scientists, and carry on their collaboration outside of the project infrastructure.

A recent study of Zooniverse discussion boards has shown that responses to comments can take many hours (or days), and that this can affect the vibrancy of a community and how it is perceived by new participants (Luczak-Rösch et al., 2014). This lack of immediacy can make asynchronous online communication less dynamic (O’Connor, 2008,

James and Busher, 2009). The absence of interaction between Planet Hunters participants was also apparent in the interview feedback, and comments referring the community (or rather the lack of it compared with other Zooniverse projects) were made by two individuals who had previously acted as forum moderators in other Zooniverse projects.

Participant observation and the feedback from surveys and interviews indicate that the level of interaction between participants is related to the project task and certain aspects of project design. A high level of task complexity means that participants interact with other more experienced members of the community if they want to actively contribute to a project. This occurs when new participants are learning about the project task, and when participants work together on the task. This is the case in Foldit and to a lesser extent among overclockers and members of the Folding@home Beta Testers as they work together to address specific hardware and software issues. The project task in Planet

Hunters is relatively straightforward in comparison to the Foldit science puzzles, and one can carry it out alone after reviewing the tutorial material provided on the website.

The presence of teams also affects interaction between participants. In Foldit and

Folding@home teams bring participants together to work on a specific project task, or to 284 compete against other teams. They can also develop their own sense of identity. Teams have their own webpages (some with their own ‘mission statements’ or approach to playing), forums (e.g. overclocking teams in Folding@home), or even their own internet relay chat channel (e.g. Foldit) where they work together in real-time. Teams on Foldit and Folding@home also have some independence from the project organisers, which may affect the dynamics and content of this interaction. The lack of teams, or an element of competition of Planet Hunters, reduces the opportunities for participants to come together, and interaction is limited to the Talk function, discussion board and project blog, all of which are overseen by the project managers and a small number of moderators.

From my observations of Foldit, internet relay chat is a key facilitator of online communication between players. Not only does it play a role in establishing collaborations between players, but it is also important for the formation of online friendships, and more general sociability (Jennett et al, 2013). Internet relay chat has also been used on one occasion by members of the Beta Testers who collaborated to address a particular software problem. The lack of internet relay chat on Planet Hunters, and the lack of a ‘quick turnaround’ with regard to responses to online posts influences the amount of online interaction between citizen scientists, and may also limit the number of participants who are motivated to interact.

8.4.2 Interaction between project team members and citizen scientists

The avenues used for communication (e.g. blog, forum, internet relay chat) and the degree to which members of the project team are ‘embedded’ and have a regular presence within the community vary between the three projects. However, the prevalence of more one-way communication from project scientists to citizen scientist

285 volunteers (particularly in Folding@home and Planet Hunters) reflects the fact that all three projects were initially organised in a top-down fashion (Mueller et al., 2012), and that volunteers were not directly involved in setting up the project or asking the research questions. This is in contrast to other types of citizen science project, where citizens, who may be concerned about an environmental issue, for example, determine the focus of the research including what questions are addressed (Irwin, 1995, Ottinger, 2010, Conrad and

Hilchey, 2011).

In Folding@home, there is some interaction between members of the project team and those in the Beta Testers (Chapter Four, Section 4.2.2). However, there were a number of points made on the online survey and the interviews that outlined a general desire for greater communication with the project team, particularly for greater visibility of the project scientists and more information about how the output of the project is used. The controversy surrounding the changes in the BigAdv points system highlighted some of these issues and some of the problems that can arise when small teams of project scientists try to communicate with many hundreds or thousands of participants, particularly when they relate to changes in the project (Chapter Four, Section 4.2.6).

In Planet Hunters, there is one key scientist who is involved with communicating with participants via Talk, the discussion boards and the project blog. This same project scientist takes an active role in communication, and considers it to be an important component of their involvement in Planet Hunters. A second project scientist takes part in research-related discussions via email with a small group of participants directly, although these discussions are not visible on the project website (and none of this group took part in the survey or interviews). From observations of the project website and from survey and interview feedback, the number of citizen scientists who interact with the

286 project scientists is very small and limited to a core group of approximately 20 very active participants.

During Foldit most of the interaction occurs during play via global and team internet relay chat. This is usually between players and seldom involves members of the project team.

Communication with the project team occurs through the project forum, the project blog

(on the Foldit homepage) and through real time online chats with members of the project team (see Chapter Four, Figure 4.4). These chats generally occur every two months and any player can take part. There are members of the project team who regularly communicate with the players and are visible on a regular basis. Interviews with Foldit developers suggest that the project team place a high level of importance on these interactions. As with Planet Hunters and Folding@home, direct interaction between members of the project team and the players is largely restricted to members of the core team.

From interview feedback and my observations, it is evident that all three project teams have experienced some problems when communicating with citizen scientists. Some volunteers can be critical of changes made to projects or of other aspects of the research.

Participants who have been with the project for many months or years, and invested a lot of their free time may develop strong opinions regarding how the project should be managed and what the priorities should be. To what extent the views of citizen scientists are taken into account, or how (and if) they are consulted when changes are made to a project may need to be considered at some point by project teams, especially if projects run for many months or years. This has been especially evident in Folding@home after the changes to the BigAdv points system was announced (Chapter Four, Section 4.2.6).

287

These observations raise issues relating to governance, and to what extent the participants are, or can become involved in influencing decisions relating to the project.

The Folding@home project team responded to concerns about the changes to BigAdv, and stated their intention to involve participants more in future decisions relating to the project. Recently, the Zooniverse team have been reconsidering the Talk function

(several Planet Hunters interviewees were critical of the vehicle for communication) and have asked participants to provide feedback. Of the three projects explored in this research, Foldit appears to take into consideration the views and experiences of participants to a greater extent, and players have been involved in helping to set the parameters of the game since its inception (Cooper, 2011). However, the Foldit team have recently experienced some negative feedback from players in response to recent changes to the project software73. Perhaps in responses to some of these issues, all three projects have recently appointed ‘community relations managers’, although their actual remit appears to vary between projects.

Issues of governance may have implications in relation to the sustainability of online citizen science projects and on the motivation of participants. Consequently, arrangements may need to evolve over time, and some projects may benefit from greater partnerships between project team members and participants, particularly those with small numbers of active participants. Some individuals invest greatly in these projects

(both in time and in financial resources), top-down governance may not remain sufficient for some of these heavily involved participants. Furthermore, other prospective participants may be put off by the lack of engagement with the project scientists.

73 Information about the changes to Foldit software: http://fold.it/portal/node/996871. Feedback from players regarding these changes: http://fold.it/portal/node/998292, http://fold.it/portal/node/998346. 288

Feedback from citizen scientist volunteers also suggests that some may have certain expectations regarding how much contact they will have with the professional scientists and become annoyed when these expectations are not met. This was apparent in some of the interview feedback from Planet Hunters participants. Project scientists may need to consider and explore the expectations of their participants and tailor their interaction accordingly, or be explicit about the role of the project team and how they intend to interact with volunteers. It also implies the existence of an unwritten social contract between the project scientists and the citizen scientists, or of a sense of mutual obligations. This mirrors more general discussions relating to ‘science and society’ in which science has a contract with wider society which is built on trust and a set of expectations of the one held by the other. Gibbons (1999) describes this contract as one where the production of scientific knowledge is seen by society as both transparent and participative. While this contract is presently ‘unwritten’ in the projects that were investigated, they could be considered and eventually ‘written’. The presence of material of this nature on a project website could help to increase trust and transparency, and may ultimately help to sustain participation.

Some of the problems relating to interaction between project scientists and citizen scientists may stem from a lack of time or personnel. All of the scientists and developers interviewed were engaged in scientific research full-time, or were involved in other projects. Many of the project team members interviewed spoke of a conflict between doing their ‘day jobs’ and trying to manage large numbers of volunteers. Practically every scientist or developer I spoke to alluded to this and stated that it was impossible to please everybody all of the time. These comments also provide an insight into how these scientists view citizen science, in particular, the degree to which they (or their

289 institutions) value the contributions of non-specialists. For example, is citizen science merely an ‘add-on’ to their research, or is it fully integrated into their ‘day jobs’? This integration of citizen science into the ‘day job’ varies between project team members.

For Foldit, it is evident from the interviews with developers that this work is central to their research activities, and observations of communication activities of scientists, as well as the research output, from the Baker Lab also suggest that this is case for the project scientists. For one of the Planet Hunters scientists, interview feedback strongly suggests that they view citizen science as central to their research efforts, and stated that this is where they saw their career continuing to develop in the future. For

Folding@home, the view of the project scientist relating to this issue was difficult to ascertain, as they were reluctant to talk about their experiences in much detail.

Recent work exploring the views of scientists involved in OPAL (Open Air Laboratories), a series of ecological and environmental citizen science projects in the UK, highlights both of these issues – the time available to engage with volunteers, and the value placed on citizen science generally (Riesch et al, 2013a, 2013b). Interview feedback from the scientists involved in this project suggests that despite the fact that many of the scientists involved viewed these projects as good opportunities to engage with volunteers, they often underestimated the amount of effort that this required (Riesch and Potter, 2013a).

The findings from my research are similar to those from OPAL. What this research adds, however, is that while the three projects provide opportunities for public engagement with research, this is not the primary motivation for scientists to become involved, and the main motivation for scientists and developers in the three projects investigated here, is a desire to employ citizen scientists to help analyse their data.

290

8.5 How can contribution to online citizen science projects be characterised? (RQ5)

Previous research into other online projects and communities has shown that there are a variety of ways in which participants can contribute (Brandtzaeg and Heim, 2009, Preece and Schneiderman, 2009, Haythornthwaite, 2009, Makriyanni and De Liddo, 2010). This research has shown that this is also the case in the online citizen science projects investigated. Participation in these projects can be considered quantitatively, for example, how long they have been contributing to the project and how many hours a week do they spend on project tasks. It can also be considered more qualitatively, for example, what else do they do apart from the main project task, are there other roles they may fulfil?

8.5.1 Contribution of citizen scientists

Of all three projects, the group of Foldit players spend the most time per week participating in the project task, with almost half (18 players) spending at least two hours a day playing. The greater time investment of Foldit players is related to the fact that the task is more time consuming than the tasks in the other projects (perhaps with the exception of the efforts of the overclockers or Beta Testers in Folding@home).

Completing a Foldit science puzzle may take several days. Learning the task in Foldit is also lengthy and complex, filtering out individuals who are less than highly committed to the game. The strength of the community (which also helps to sustain participation) and the fact that many collaborate and co-operate together on the science puzzles may contribute to the greater level of time commitment demonstrated by some Foldit players

(Borda and Bowen, 2009, Preece and Shneiderman, 2009).

291

As well as quantitative differences in the way an individual participates, this research has also identified other project-related tasks. These tasks have largely been identified through participant-observation, and confirmed by survey and interview feedback. Table

8.3 lists additional ways that an individual can contribute to each project. Some of these extra roles require specific skills such as knowledge of computer hardware or software, or language skills (for translation).

Table 8.3 Additional project tasks and roles

Foldit Folding@home Planet Hunters Forum / internet relay chat Forum moderators Forum moderators moderators Translating web content Translating web content Translating web content (recent request for help with this by Zooniverse team) Team co-ordinators Team co-ordinators Teachers / instructors (help Beta Testers member new players) Technical experts (e.g. project software) Scripters (write ‘recipes’ ) Strategy specialists (e.g. hand folders, soloists and evolvers)

Foldit has the greatest diversity of tasks and many of these have evolved as a result of the complexity of the task and the learning process. Most of the additional roles in

Folding@home are related to co-ordinating the efforts of the Beta Testers and teams of overclockers. In the absence of teams, competition and with a relatively straightforward project task, the diversity of roles is reduced in Planet Hunters. Offering a diversity of tasks may help to sustain motivation in online citizen science projects, and this aspect of participation requires further investigation.

8.5.2 Patterns of participation and contribution

Each project has a small community of very active participants, who are a smaller proportion of the population of registered participants. This research found that among 292 the groups of active participants, are smaller groups of highly dedicated volunteers, or

‘core’ participants. They are defined as participants who:

 go beyond the project task and take on other roles within the project (see Table

8.3);

 complete a far greater number of project tasks than other participants; and

 use their involvement in the project to ask their own research questions or

undertake their own independent research (this can occur independently or with

some help from the project scientists).

In Foldit, these individuals have created and filled a range of roles in addition to dedicating many hours a week to the project. Some of these individuals have been with the project since its launch in 2008. In Folding@home, core participants were found among members of the Beta Testers, and among the more active teams and communities of overclockers. In Planet Hunters, there are a small number of individuals (estimated to number 20) who do much of the classification work. Some carry out their own analysis using tools provided by the scientific team or that they have developed themselves.

These individuals are highly visible on the discussion boards, and two of them have been named as co-authors in several Planet Hunters publications (Wang et al., 2013, Schwamb et al., 2013). Small groups of core participants can create a ‘community of practice’ (see

Chapter Four, Section 2.4), and members of these communities share a repertoire of resources, experiences and skills (Wenger, 2006). As a result, these groups have close inter-personal ties (McLure Wasko and Faraj, 2005).

The ‘reader-to-leader’ framework

The ‘reader-to-leader’ framework (outlined in Chapter Two, Section 2.5) is of relevance to the patterns of contribution observed in these online citizen science projects, and like 293 other online communities, there are decreasing numbers of participants with increasing levels of commitment and involvement (Preece and Shneiderman, 2009). This framework also illustrates that users don’t always progress from one stage to another, and there can be movement in both directions between the different levels of participation. Similar observations have been made in this research in relation to the three projects investigated. For example, in Planet Hunters and in Foldit, contribution can wax and wane as participants may not participate for a while but become more involved when they have free time. However, the ‘reader-to-leader’ framework does not make it clear that progression along this ‘continuum’ doesn’t necessarily mean that the participant no longer engages in other types of participation (e.g. a leader may still be a reader, and a contributor or collaborator). With this in mind, the framework has been adapted for online citizen science projects, and is based more on a Venn diagram that illustrates that level of contribution sits within a wider participatory context (Figure 8.1).

Each project has a large number of registered users, in effect, the potential pool of contributors. From this group of registered users, a smaller number make a limited number of contributions, or contribute for a short period of time. These individuals have been referred to transient participants (Ponciano et al., 2014) and as “dabblers” (Jennett et al., 2014) and they contribute on a more ad hoc, or casual basis, spending short amounts of time on a project task when they have the time and inclination. Individuals in this group are less likely to interact with other participants (or project scientists), or get involved in other project roles. This pattern of participation was reported by most (75%) of the Planet Hunters survey respondents, and it has been observed elsewhere on the

Zooniverse (Ponciano et al., 2014, Jennet et al., 2014).

294

Figure 8.1 Adaptation of the ‘reader-to-leader’ framework for online citizen science projects

From the population of registered users, a smaller number of participants will become interested in the project and participate on a regular basis and perhaps over a longer period of time. These individuals are more likely to become involved in the more social aspects of the project, and in other project roles. The active playing community in Foldit is an example of this type of contribution. This group of active participants may show this level of commitment from the beginning of their involvement in a project, or they emerge from the group of more casual participants. Conversely, active participants may reduce their level of contribution and move to the group of more transient participants. This is illustrated on Figure 8.1 as a ‘transition zone’ and illustrates that there can be some movement between these two groups of participants. Several Planet Hunters

295 participants who were interviewed, spoke of varying their involvement in the projects based on the amount of free time they had available.

Out of the group of active participants, will emerge a number of core participants. These citizen scientists are more likely to interact with each other and with members of the project team. They may work together either co-operatively or collaboratively. Core participants are more likely to get involved in other project-related tasks such as moderating forums, or mentoring new participants. Core participants do not emerge from the group of transient participants or ‘dabblers’, as they require an in-depth knowledge of the project and the related tasks, something that is more likely to be acquired during active participation.

One thing that should be emphasised is that while different levels of contribution may exist, as well as different ways to contribute, some individuals will be quite happy to remain as more casual participants or dabblers. Not everyone has the inclination, time, or in some cases, the skills to be a core participant. This was especially evident in the feedback from Planet Hunter participants, and many liked the fact that they could dip in to the project when they had the time. Indeed, one of the Planet Hunters scientists made the point that both types need to be catered for, the dabbler and more committed participant, and this may well be one of the more challenging aspects of working with groups of citizen science volunteers.

Lightweight and heavyweight peer production

Another way of considering patterns of participation in online citizen science activities is through the lens of Haythornthwaite’s (2009) model of lightweight and heavyweight peer production (see Chapter Two, Table 2.3). The results of this research suggest that a continuum of lightweight and heavyweight behaviours (rather than a dichotomous 296 classification) may be more appropriate with regard to online citizen science, as projects exhibit a range of both types of characteristics. Furthermore, different types of behaviours may be observed within one project. Figure 8.2 illustrates a continuum of lightweight and heavyweight behaviours, and identifies the location of groups of project participants.

In ‘lightweight peer production’, individuals can easily contribute, and there is usually a large set of participants (the crowd) who provide minimal additions to the endeavour as a whole. The ‘rules’ of contribution are defined by authorities or owners of such projects, and participants are not expected to play a role in determining the direction or the project as a whole. Participants do not need to make long-term contributions, nor do they need to interact with others. The more casual participants of Folding@home (e.g. those who just download and run the project software), would fall into this category, as would (to a lesser extent) the ‘dabblers’ of Planet Hunters (see Figure 8.1).

In heavyweight peer production, success depends upon a critical mass of contributors

(the community) who make significant time investments to the project and who interact with other participants in order to sustain the community. There are learned norms of interaction and language which are indicative of community membership. Outsiders or novices can be easily identified. This applies to the Foldit community (especially the core group), the Beta Testers and overclocker communities in Folding@home, and the small number of core participants in Planet Hunters.

297

Figure 8.2 Lightweight and heavyweight scales of peer production and position of the three online citizen science studies investigated in this research

Findings from all three data streams of evidence used in this research have shown that citizen scientists are not a homogenous group. Both the ‘reader-to-leader’ framework

(Preece and Schneiderman, 2009) and the work of Haythornthwaite (2009) enable a further consideration and illustration of this observation. These frameworks also help to place online citizen science within the broader context of open online contribution by illustrating that the patterns of contribution observed are not unique to online citizen science.

8.5.3 Contribution of project team members

Interviews with scientists and developers suggest that there are a number of ways that members of the project team can contribute to online citizen science projects, although their main contribution is their scientific and technical expertise. In these examples, project scientists set the research parameters, decide the research questions and utilise 298 the output of the projects. Online citizen science projects also require the input of software developers, games developers, and website designers. Indeed, the technological input into these projects is of great importance, and contributes greatly to the potential success of a project (Prestopnik and Crowston, 2012).

Project team members must also promote and communicate about the projects in order to recruit more participants. Thus, scientists and developers can also contribute by participating in these communication efforts. From interviews with project teams, and through participant-observation, it appears that the degree to which this occurs can vary greatly. Some project scientists and developers are very active in talking to external audiences and promote their projects at academic conferences or to journalists (e.g.

Foldit, Zooniverse projects). Some appear to rely more on ‘word of mouth’

(Folding@home), and their involvement in promotion is on a much smaller scale.

Members of the project team can also make a contribution through their interaction and engagement with members of the community of citizen scientists. Some participants are very interested in the science behind the project and some report enjoying the interaction they have with professional scientists. As outlined in Section 8.4.2, this research has shown that the extent to which this occurs, and how it occurs, can vary from project to project. Some project team members greatly enjoy this aspect of their involvement in online citizen science, while for others, it can be more challenging.

8.6 How do participants perceive their role in the project? (RQ 6)

How participants view their contribution to online citizen science projects has not been previously explored in any detail. Views regarding individual contribution, particularly with regard to their involvement in scientific research varied greatly and a wide range of opinions were expressed. However, the majority of respondents (n=356, 63%) felt that 299 they were making a contribution to science, although the importance they placed on their individual contribution varied. Some felt very strongly that their contribution was important, if not vital. This was especially evident in the interview feedback with Foldit players and Folding@home participants who are members of the Beta Testers.

“It [Beta testers] contributes a lot. Without it, f@h would not be where it is today. We would not be able to use our hardware to the fullest. In recent years Pande Group with the help of beta testers brought in huge amount of fantastic innovations, which improved project immensely.”(FH4)

“I am involved in scientific research. I approach it as such. If this really was just a game I would stop today and dedicate my intellect, time, money and determination to a more worthy cause.” (FD8)

Approximately a quarter of folding@home and half of Planet Hunters survey respondents felt their contribution was small but significant when pooled with the efforts of other volunteers. Foldit players tended to view their contribution more positively in terms of individual impact, while Planet Hunter respondents felt that their contributions were more of a “drop in the bucket”.

This illustrates that the complexity of the project task may be related to one’s perception of contribution. Those who invest more time in learning a difficult task, and carrying it out may feel more strongly that they are making a more important contribution to a project than someone who occasionally spends time on relatively straightforward classification task. Projects that have a greater diversity of tasks and give participants the opportunity to get involved in other ways may also influence the way participants feel

300 about their contribution (this may have been important for Foldit players). However, the role of task diversity requires further investigation.

How the members of the project team view or value the involvement of citizen scientists can influence how the volunteers feel about their contribution. For example, approximately a quarter (n=102) of Folding@home participants do not feel as though they are involved in scientific research, and 57 of this group felt they were just donating their resources (even though there were some overclockers and hardware enthusiasts in this sub-group) and likened their involvement to making a financial contribution. This view of donation rather than contribution could have been reinforced or promulgated by the fact that the project team refer to the Folding@home participants as ‘donors’. This is contrast to Planet Hunters where participants are referred to as collaborators. The fact that Folding@home participants are not acknowledged in publications based on the output of the project could also be of importance regarding this issue. However, only 20 survey respondents (5%) mentioned that they would like to see greater acknowledgement of participants on resulting publications.

How an individual views science and what they think scientific research actually entails may influence views about their own participation in a project. One of the Planet Hunters scientists referred to “Hollywood science” implying that some of the project participants may not have an accurate grasp of what actually constitutes scientific research in the ‘real world’. While giving citizen scientists an experience of authentic scientific research has been cited as one of the benefits of their involvement in citizen science projects (Bonney et al., 2009, Cronje et al., 2011, Riesch et al., 2013a), this issue may require additional consideration. Many of the participants in this research have formal qualifications in

STEM subjects which may increase their level of awareness of the scientific research

301 process. However, unless an individual is a professional scientist (and very few respondents in this study are), they may, over time, become less familiar with the processes involved in carrying out scientific research. From these results, the complexity of the task and the views of the scientific team appear to be more important. However, further research that explores the views of citizen scientists about scientific research would be of interest and help to address this question in greater depth.

Project team members view their contribution in terms of the responsibilities it entails, and how well they achieve their goals. All of those interviewed stated that their main role was to ensure that valid and high quality scientific research was being carried out. The

Planet Hunters scientists were keen to stress that they should not be wasting anybody’s time, and that they had an obligation to produce something meaningful from the efforts of volunteers. The developers of Foldit also spoke about their efforts to produce an entertaining game that would appeal to current and potential participants. The ‘quality’ of their efforts was linked to the number of scientific publications produced. Such comments not only reflect their views concerning their contribution to a project, but are also a reflection of the primary motivation for their involvement, which is to carry out and produce valid and useful scientific research.

8.7 Conclusions

Overall, it appears that the three projects appeal to self-selecting groups of people.

Respondents tend to be male, well educated (often in STEM subjects) and have an interest in other science-based activities. Why these projects may appeal more to men or to ‘confident engagers’ of science has been considered. Aspects of project design, subject content, and where the projects are promoted may influence who they ultimately attract as participants (Cooper et al., 2010, Jennett et al., 2014). Other socio-cultural factors,

302 such as demographic characteristics and socio-economic background, may also influence who is attracted to these projects (Dawson and Jensen, 2011).

In all three projects, making a contribution to science and a background interest in science or computing were the main reasons that the citizen scientists initially joined.

Factors sustaining participation showed greater variation between the projects and appeared to be related to the project goals and the level of difficulty of the project task.

The motivations of citizen scientists are dynamic and can change over time (Batson et al.,

2002, Rotman et al., 2012). Scientists primarily become involved in online citizen science projects in order to accomplish research involving the analysis of large datasets. Some also see these projects as an effective way to engage ‘the public’ and become more involved in communication activities. However, working with large communities of volunteers can be problematic for members of the project team, especially when changes are made to the project. Issues relating to governance and communication with large groups of volunteers have been highlighted, and demonstrate that communities of volunteers need to be supported, and may want to be consulted when changes are made to a project.

Each project offers a number of opportunities for online interaction between participants.

The presence of internet relay chat, and forums with a quick ‘turnaround’ of comments facilitate greater interaction between citizen scientists, and may help to sustain participation in projects where the task is complex (e.g. Foldit). A variety of roles were available to citizen scientists in each project, although a greater diversity of tasks was identified in Foldit. All three projects have groups of highly committed participants, or core participants, who do much of the ‘work’. They interact more with each other online than less-committed participants, and they also interact more with members of the

303 project team. This pattern of participation has been observed in several other online citizen science projects (Ponciano et al., 2014, Jennett et al., 2014). The groups of core participants can be considered as the ‘leaders’ of these project communities, as defined by Preece and Schneiderman (2009). However, this asymmetric pattern of participation means that scientists in all three of the projects have to cater for varying levels of interest and involvement.

Most of the respondents felt they were making some kind of contribution to science, although views about personal contribution varied both between and within projects.

Differences in the view of one’s contribution may be related to the complexity of the task, diversity of project tasks and the views of the project team regarding the work of citizen scientist volunteers.

This analysis has attempted to draw together the main themes that have emerged from the three streams of data, and to identify explanations and frameworks which help to elucidate the findings. In addressing the research questions I have compared the three projects and identified points of similarity and divergence. Despite very different project tasks there are some important common features that relate to motivation to participate, and to the fact that there are different ways that citizen scientists can contribute. One of the most significant findings of this research has been that relatively small groups of citizen scientists are needed to make these projects viable. Despite attracting interest from many thousands of potential participants, only a fraction of these individuals will actively contribute. The commitment of these small groups of active and core participants makes these projects successful, and enable the realisation of the research goals.

304

Chapter 9: Conclusion A review of literature from a range of academic fields has highlighted a number of gaps in our understanding of online citizen science projects (Holohan and Garg, 2005, Raddick et al., 2010, 2013, Krebs, 2010, Nov et al, 2011a, 2011b, World Community Grid, 2013). This research has attempted to address these gaps by identifying and elucidating six research questions. A mixed-methods case study approach has been employed and a comparative analysis of three selected projects has been undertaken (Thomas, 2011a).

Patterns of online interaction, contribution, and motivation to participate have been investigated in relation to both citizen scientist volunteers, and the scientists and developers who set up and manage the projects. Several theoretical models have been employed to examine motivation to participate in the selected projects that were originally derived from studies on general community volunteering (Batson et al., 2002), formal education (Ryan and Deci, 2000, 2009) and participation in open source software

(Hars and Shaosong, 2002). Two other theoretical models, the ‘reader-to-leader’ framework (Preece and Schneiderman, 2009) and the ‘lightweight-heavyweight’ model of peer production (Haythornthwaite, 2009), have been used to characterise participation and contribution.

The research questions have been addressed through findings from online surveys, semi- structured interviews, and through an analysis of my experience as a project participant.

The findings have been presented in relation to the research questions in Chapters Five through Eight.

305

9.1 Contributions of this work

There are several ways in which this research has made contributions to knowledge. In addition to the production of new knowledge relating to online citizen science (including demographic information), the inter-relationship between several parameters of participation (namely, motivation, contribution and interaction) has been explored. The typology of online citizen science projects presented in Chapter One (see Table 1.1) has been reconsidered, and a new typology based on project task (rather than project ‘type’) has been formulated (Table 9.1). Using a mixed-methods case study approach has resulted in several methodological insights that may provide a basis for future work in this area. Finally, the findings of this research have been used to generate some considerations for scientists and developers who may be thinking about setting up an online citizen science project.

9.1.1 Contributions to knowledge

New knowledge has been generated over the course of this research which ultimately enhances our understanding of online citizen science projects. These contributions to knowledge have been grouped according to which research question they help to address, and can be summarised as follows.

RQ 1 Who participates in online citizen science projects?

 While many individuals register to participate in a project, only a small proportion

become active participants and contribute on a regular or sustained basis (see

Chapter Four, Sections 4.1.3, 4.2.4, and 4.3.2). What constitutes an active

participant varies between projects and is related to productivity and effort. The

distinction between active and inactive is not absolute; these categories are

blurred. In order to fully explore and understand online citizen science projects,

306

the role of active participants must be considered. This will inevitably entail

identifying who these individuals are. The implication of this finding is that the

organisers of online citizen science projects will need to attract a large pool of

registered participants, in order to yield a more committed group of active

participants.

 This research has added to the small body of demographic information about

participants in online citizen science projects (see Chapter 5, Section 5.1.1;

Chapter Six, Section 6.1.1; and Chapter Seven, Section 7.1.1). As in previous

studies, I have found an over-representation of male participants (Holohan and

Garg, 2005, Raddick et al., 2010, 2013, Krebs, 2010, Estrada et al., 2013, World

Community Grid, 2013). Survey feedback has also highlighted that these groups of

active participants are well-educated (often with a formal STEM qualification),

have an interest in science more generally, and are confident users of ICT (see

Section 8.1, Chapter Eight). These findings have implications with regard to

widening participation in these projects, and those involved in setting up and

managing them may wish to consider if they wish to attract a greater diversity of

participants.

RQ 2 What motivates and sustains participation in online citizen science projects?

 The findings indicate that, typically, citizen scientists begin participating in a

project in order to make a contribution, and because they are interested in the

science and goals of the project (see Chapter Eight, Table 8.2). Motivation to

participate is dynamic and motivations that sustain participation can be different

from those that initiate it. Participants can become de-motivated if they feel their

contributions are not valued or if major changes are made to a project without

307

any consultation. Thus, decisions made by project teams relating to

communication and governance, play an important role in sustaining motivation.

 Project scientists and developers are motivated to take part in online citizen

science projects in order to get help with research tasks, or to explore the

potential of computer technology (such as games) in solving scientific research

problems. Projects enable scientists and developers to interact and engage with

participants, and provide non-specialists with an authentic experience of scientific

research. However, the degree to which these opportunities for engagement are

taken up varies between the projects, and between different individuals within

project teams.

RQ 3 Do motivations vary between different types of online citizen science projects and their associated tasks?

 A desire to make a contribution to scientific research was the key motivator that

initiated participation across the three projects. This supports the findings of

previous research (Holohan and Garg, 2005, Raddick et al., 2010, 2013, Krebs,

2010). This research builds on previous work by demonstrating that motivations

that sustain participation show a greater variation between projects, and are

related to aspects of the project task, such as level of difficulty, the presence of

competition, and opportunities for discovery (see Table 8.1, Chapter eight).

Understanding the motivations that sustain participation, and the de-motivations

that have the opposite effect, have practical applications for those wishing to

sustain a project over time.

308

RQ 4 How do project participants interact online?

 The extent to which participants interact is influenced by the complexity of the

project task. Difficult project tasks necessitate greater interaction between

participants as they help one another learn about the project task, or work

together to complete the task (see Section 8.4.1, Chapter Eight). The presence of

technical features such as internet relay chat, facilitates synchronous

communication between participants, and can help to create a dynamic online

environment. Asynchronous forums enable communication between participants

who are geographically and temporally dispersed. However, both of these vehicles

for communication entail technical and infrastructure requirements for both the

participants, and the project organisers. These communication channels also

require moderation. The presence of a competitive element can also increase the

level of interaction between participants as they assemble and co-ordinate

themselves into teams.

 Regular communication with members of the project team is important for some

citizen scientists, and there are a number of approaches that can be taken. In

some cases, there may be a designated individual (e.g. a community support

manager) who acts as the interface between the project team and community of

participants. Interacting with communities of citizen scientist volunteers can be

difficult at times, and some underestimate this aspect of their involvement, and

the time commitment it entails. Issues relating to communication have

highlighted the importance of governance in online citizen science, and illustrate

that the degree to which volunteers are consulted as projects change and evolve

over time, and can vary between projects (see Section 8.4.2, Chapter Eight).

309

RQ 5 How can contribution to online citizen science projects be characterised?

 Small groups of core participants emerge from the wider community of active

participants who show a high level of commitment to a project. These individuals

can co-operate and collaborate while carrying out a project task, and may

ultimately create online communities of practice. These small groups of

committed volunteers help to sustain these projects, often carrying out a large

proportion of the project tasks, and filling a variety of roles (see Figures 8.1 and

8.2).

 Online citizen science projects can offer participants a number of ways to

contribute in addition to the main project task (see Chapter Eight, Table 8.3).

Other tasks identified through the course of this research include moderating

forums, co-ordinating and managing teams, fixing and identifying software

problems, instructing new participants, translating web material, and developing

new approaches relating to the project task (such as writing ‘recipes in Foldit).

 Not all participants want to become core members, get involved in other project

tasks, or even interact with other participants. It would appear that certain types

of project task (such as the task in Planet Hunters) allow participants to participate

more sporadically and in isolation, but still in a productive way (see Chapter

Seven, Section 7.1.3). Online citizen science projects also create opportunities for

these individuals to be involved in authentic scientific research.

RQ 6 How do participants perceive their role in the project?

 There is some variation in how citizen scientists view their personal contribution

to a project, and the degree to which they feel involved in scientific research. For

some scientists and developers, their own contribution or degree of ‘success’ is

310

measured in terms of useful scientific output or results. There is also some

variation in the way project team members view their relationship with citizen

scientist volunteers and this is evident in the language they use to describe

participants. For example, they have been referred to as ‘donors’ in

Folding@home and as ‘collaborators’ in Planet Hunters. The language used to

describe participants may be indicative of an existing ‘hierarchy’ within a project,

and the extent to which there is a potential for a more meaningful and

collaborative partnership to develop between scientists and citizen scientists.

These findings suggest that online citizen science projects have been important in making scientific research more open for a number of distributed volunteers. These individuals have responded to the challenges presented by these projects, increasing their scientific and technical knowledge, and self-organising into various roles and teams in order to produce new knowledge (Khatib et al., 2011, Eiben et al., 2012, Schwamb et al., 2013,

Wang et al., 2013). This increased openness of scientific research can be placed within the wider context of changes in ‘science and society’, as science becomes more transparent, contextualised, and participative (Gibbons, 1999, Miller, 2001, Bauer et al.,

2007). Online citizen science projects have mirrored some of these changes enabling (and empowering) interested citizens to become more involved in the generation of new scientific knowledge.

311

9.1.2 The inter-relationship between contribution, motivation and interaction

Through the course of this research, it has become evident that motivation to participate, interaction with other participants, and the type of contribution, are inter-related to some degree, and that these relationships are mediated by the project task. Figure 9.1 summarises these relationships, and is based upon the findings from the three data streams.

Figure 9.1: Inter-relationship between contribution, motivation and interaction

The nature of the project task (e.g. is it in an area of science that is of interest, or is the task considered appealing in some way) motivates participation in a project (A). The level of complexity of the project task influences the level of interaction between participants, and the greater the complexity of the project, the greater the likelihood that participants 312 will learn from other participants, or co-operate and collaborate with others (B). If the task has a competitive element, this will also facilitate interaction by encouraging the formation of teams (Foldit and Folding@home). The amount of time participants devote to a project and the diversity of roles available is influenced by the complexity of a project task (C), and projects with a complex task (such as Foldit) stimulate the development of a greater number of related tasks, and may require a greater time commitment from participants in order to learn and carry out the task (e.g. Foldit).

Motivation, interaction and contribution are also inter-connected. A high level of interaction between participants, or with members of the project team can be a powerful motivator and the presence of an online community can help to sustain participation (D).

Conversely, highly motivated participants may be more likely to interact with other participants, and want to work or share their experiences with others.

Interaction with others may stimulate a greater level of contribution to a project (E) by creating a requirement for other project roles (e.g. moderation of forum or online chat) and by facilitating collaboration and co-operation between participants. The more a participant contributes to a project, the greater the likelihood that they will come into contact and interact with other participants, or members of the project team.

Highly motivated participants may be inspired to make substantial contributions to a project, both in terms of their time commitment, and perhaps their involvement in other project roles (F). Being able to contribute to a project in number of different ways, may motivate an individual to sustain participation. If this contribution is felt to be of importance and is valued, then this too may motivate partipation.

313

The inter-connection between these aspects of participation has not been considered in previous research, and the elucidation of these relationships was made possible by a detailed examination and comparison of the three selected projects made possible by adopting a case study approach.

9.1.3 Methodological insights

A mixed methods case study approach has been used to investigate three online citizen science projects. This approach allows a researcher to examine a phenomenon in depth and from a variety of angles (Gillham, 2000, Yin, 2003, Thomas, 2011a). While this approach is not ‘new’ and has been applied in other areas of study, it has not been used in previously published research in online citizen science.

By utilising a case study approach, I have been able to investigate several aspects of participation in detail, including contribution, interaction, and motivation. It has also enabled an exploration of how these parameters may be inter-related both within and between projects (see Figure 9.1). The insights gained from this aspect of the research would not have been possible if only one facet of the projects was considered (e.g. motivation).

Another advantage of using a case study approach has been that it has facilitated a comparison between three different projects. In each case, a standardised approach to data collection and analysis was employed, which has made the results more comparable across the projects. Online citizen science projects are well-suited to a case study approach. There are numerous data strands that can be examined in addition to the ones considered in this research such as transcripts of forum discussions, podcasts, and external press articles. This abundance of data also presented challenges, as I had to select the most relevant data strands that would help me address my research questions. 314

Most of the previous research on online citizen science projects has made use of quantitative surveys containing Likert scales (Holohan and Garg, 2005, Krebs, 2010, Nov et al., 2011, Raddick et al., 2013, World Community Grid, 2013). While this approach facilitates quantitative statistical analysis, my approach has been to utilise a greater number of open-ended questions in my online surveys. As a result, the responses that I have obtained from participants (particularly with regard to motivation) appear to have been more wide-ranging than those reported in previous research (see Chapter Eight,

Table 8.2). This suggests that this approach may be able to encapsulate a greater diversity of responses and viewpoints which may not be identified in more prescriptive questionnaires (Jamieson, 2004, Pell, 2005). Using more open-ended questions on the first (Foldit) survey also helped me to identify another aspect of participation that I had not previously considered: how participants viewed their contribution to the project. This was subsequently included on the Folding@home and Planet Hunters questionnaire.

Becoming a participant in all three projects has helped to provide a deeper insight into how participants interact online, who interacts, the identification of key and committed participants, and an understanding of what is actually entailed in the project task (DeWalt and DeWalt, 2002, Kawulich, 2005). It informed various stages of my research (see

Chapter Three, Figures 3.3 and 3.4), and through participation in these projects, I have made contact with numerous members of the citizen scientist communities. Some of these individuals have been important in providing information about the projects including issues and developments which may have not been immediately apparent by just making observations of the project website (e.g. the controversy surrounding the changes to the BigAdv points system in Folding@home). The utility of this approach also

315 became apparent during my investigations of important sub-communities of participants such as overclockers, and the Foldit core group.

Overall, using a mixed methods case study approach has permitted a close examination of the three projects, as well as a detailed comparison. Using different data streams has provided multiple sources of evidence with which to address my research questions, and has facilitated methodological triangulation. Using participant-observation as one of these data streams has provided an insight into each of these projects, and an appreciation of what participation involves, and the extent to which some individuals commit to these endeavours. I would recommend this approach to other researchers who are interested in investigating online citizen science projects, or other interactions mediated by, and through technology.

9.1.4 Revisiting the typology

In Chapter One (Table 1.1) a typology of online citizen science projects was presented and used as the basis for the selection of projects as case studies. It consisted of three different ‘types’ of project: distributed computing, distributed thinking, and citizen science games. The findings of this research suggest that this typology should be reconsidered and that it may be more appropriate to classify citizen science tasks rather than projects. Table 9.1 outlines three main types of task associated with online citizen science projects, a description of the task, the level of contribution or commitment required, and some key examples.

316

Table 9.1 Revised typology - online citizen science project tasks

Type of project Description of task Level of contribution Examples task DISTRIBUTED Provision of ‘Passive’ Folding@home, COMPUTING computer processing participation, World Community power. programme is Grid, SETI@home. downloaded and run. DISTRIBUTED Classification, Participation can be Zooniverse projects, DATA ANALYSIS annotation, transient (‘dabblers’) Stardust@home, transcription of text, or more regular and citizen science game interface. sustained. games (EteRNA, Foldit and Phylo). DISTRIBUTED Participants work Active and Foldit core players, COLLABORATION together to committed Planet Hunters core complete the participation. Groups participants, project task or other of core participants Folding@home Beta related tasks. can create online Testers, Collaboration can communities of overclocking teams involve members of practice. in distributed the project team. computing projects.

Distributed computing remains as a category of task, however, in this new typology, it refers specifically to a more ‘passive’ type of participation, where individuals simply run the project software. Distributed data analysis involves classification, annotation and transcription of data (as seen in the projects of the Zooniverse). It also includes online citizen science games as a type of distributed data analysis (albeit through a graphical games interface) rather than as a distinct category. Participation in distributed data analysis may be on a more transient basis (as demonstrated by many of the Planet

Hunters survey respondents), or it can be more active and sustained.

In distributed collaboration, participants can work together to address the project task, or a related project task such as improving the project software (Paulos, 2005, Cranshaw and

Kittur, 2011). Those who engage in this task are usually more committed, and often includes core participants, and in some cases, members of the project team. Distributed

317 collaboration can occur in any online citizen science project, and in this research, it can be seen in the Foldit core group, the Planet Hunters core group and in the Folding@home

Beta Testers.

One of the main findings of this research has been that, as with other types of online projects or endeavours, there are different ways in which an individual can participate in online citizen science (Preece and Schneiderman, 2009, Haythornthwaite, 2009).

Therefore, different types of task may be observed in one project. This typology takes this factor into consideration, and thus, an individual project may make more than one appearance in this typology. However, all types of project tasks are ‘distributed’ in that participants do not have to occupy the same geographical space or time zone (Holliman and Curtis, 2014).

While this typology is based on research relating to three specific online citizen science projects, it can be applied to tasks associated with other current online citizen science projects (Jennett et al., 2014, Lee et al., Tinati et al., 2014), and may therefore be of utility to researchers in this area who are interested in aspects of participation and contribution.

9.1.5 Considerations for scientists and developers interested in online citizen science

The number of online citizen science projects has increased significantly over the past ten years and thousands of individuals have been active participants between 2004 and 2014

(Roy et al., 2012, Gura. 2013). These projects have enabled scientists to process large amounts of data, utilise the diversity of human pattern recognition and problem solving skills, and on occasion, have resulted in contributions to knowledge (Lintott et al., 2009,

Khatib et al., 2011, Lane et al., 2013, Schmitt et al., 2014). The findings of this research,

318 suggest that those interested in setting up such a project, including scientists, developers, and also experienced citizen scientists, could usefully consider some of the following.

Varying levels of commitment

Citizen scientist volunteers will show varying degrees of commitment to, and involvement with, a project (see Figure 4.19, Chapter Four). Some may want to participate occasionally, while others may seek greater opportunities to become involved in the research. A project interface therefore, needs to be able to cater for more transient participants (or ‘dabblers’), as well as those who become core contributors (see Chapter

Four, Figure 4.19).

Project task

The level of difficulty of a project may affect the number of active participants (e.g. a more complex task may attract fewer participants), and may also affect the degree to which participants interact. More complex tasks may mean that new participants need help from others to learn them, or that participants may need to collaborate and co- operate in order to carry them out.

Facilitating interaction

Some individuals like to interact with other participants online and the presence of a project community can help to sustain participation. There are a number of ways that online interaction, both between citizen scientists, and between citizen scientists and members of the project team, can be facilitated. This can include forums, blogs, and synchronous internet relay chat. Introducing a competitive element, and enabling the formation of teams, can also promote interaction between participants. When and how members of the project team are going to communicate with participants needs to be

319 made clear. Communication with scientists can help to motivate and sustain participation, and make participants feel that their involvement is valued.

Demonstrate the value of the efforts of volunteers

Individuals take part in these projects and remain committed for a variety of reasons.

Being able to make a contribution to science appears to be of importance, and project team members may need to demonstrate the utility and value of the efforts of citizen scientists. In addition to regular communication with citizen scientists, this can be achieved through the publication of research results and public acknowledgement of their effort (Lintott et al., 2009, 2013, Khatib et al., 2011, Schwamb et al., 2013, Lee et al.,

2014). Volunteer citizen scientists value recognition for being productive, and this helps to sustain participation in a project.

Investing time in the community

Some projects attract many hundreds or thousands of registered participants. Interacting with these communities of volunteers can be demanding and should not be underestimated. Some participants want and expect some interaction with professional scientists as part of their involvement in the project, and scientists and developers may need to be explicit about how (and when) they interact in order to manage the expectations of the community. Supporting communities of volunteers is time- consuming, but it is important with regard to motivating and sustaining participation, and in promoting transparency.

Governance

Issues of governance should be considered in relation to the sustainability of the project, and in terms of the transparency of communication. Individuals who make a substantial personal investment (and financial investment in some cases) may want to become more 320 involved in decision making, or at least, be consulted when major changes are made to the project, as their involvement progresses over time. As scientists are initially involved in setting up these projects and in determining the research parameters, online citizen science projects appear to be mainly top-down in terms of their governance and organisation (Mueller, 2012). However, opportunities may arise for greater collaboration with citizen scientists, and this may help to motivate and sustain participation.

9.2 Limitations of this study

While efforts were made to ensure that appropriate methods and methodological approaches were utilised, there are some limitations of this research. While the use of a case study approach has enabled a detailed consideration of the projects in question, a selection of three completely different projects may well have provided a very different picture regarding motivation, contribution and interaction. Different strands of evidence were considered in this research, and the methods adopted for each are subject to certain limitations (see Chapter Three, section 3.3.2).

In total, three strands of evidence were considered: online surveys, semi-structured interviews, and participant observation. Alternative strands of evidence could have been considered in addition to, or instead of, the ones selected, e.g. project-related web content such as project wikis, FAQs, podcasts, transcripts of online ‘chats’ between and discussions (see Section 9.1.3). These may have contributed to a greater appreciation of online interaction and some of the alternative tasks that a small number of participants become involved in. However, the final selection of the three sources of data was driven by the focus of the research questions, as well as time constraints which necessitated the prioritisation of the analytical approach.

321

For each project, only a small number of participants took part in the surveys and interviews. It is unlikely that these groups of respondents are representative of the total population of registered participants. While this research has ultimately focussed upon the population of active participants, it is unable to provide information about the characteristics and behaviour of participants who are less active, or inactive. There is also the possibility that my sample of respondents was not representative of the wider population of active participants. This may be particularly true of Folding@home, where the sample was a much smaller percentage of the estimated population of active participants (which in itself, was difficult to estimate). The small number of survey respondents has meant that little quantitative analysis of the data was possible. Small sample sizes have made it very difficult to obtain any statistically significant results (see

Chapter Three, Section 3.4.1).

9.3 Future work

While this research has made contributions to knowledge regarding certain aspects of contribution and participation in online citizen science projects, there is substantial scope for further work in this area.

This particular study has provided a snapshot into three different projects. It would be of interest to explore how they develop over time, how numbers of participants vary, and if a project has a natural lifespan. What factors contribute to the longevity (or otherwise) of an online citizen science project? This would help to contribute to discussions relating to the sustainability (and concomitantly, funding) of projects.

Interviews with some Planet Hunters participants also suggest that personal involvement in projects waxes and wanes, and that after time, the more committed participants can desert a project and move on to something new. Of the three projects I have 322 investigated, Foldit would perhaps be the most likely candidate for a further exploration of this issue, as it is relatively easy to track the contribution and activity of individual players through the leader boards, player profile pages and participation in online chats with the project team. The contributions and experiences of a small number of project participants could be studied in detail over time.

Given that this research has focussed on active participants, a further investigation into less active or inactive participants would be of interest. For example, why do those who show some initial interest in a project, then decide not to make any contribution? What factors play a part when an individual is making the decision to switch between projects, or withdraw? This may have implications for the design of a project (Newman et al.,

2012).

Interview and survey feedback from citizen scientist volunteers regarding their contribution to these projects has raised questions relating to their views about science and what constitutes scientific research. A further exploration of these views would be of interest, and may help increase understanding of motivations relating to ‘making a contribution’ and what participants’ expectations of their involvement may be.

Understanding views about science could be useful to project managers and may in turn influence the design or interface of a project (e.g. how might a task be positioned within the overall research design), or how the project team interacts with participants.

The presence of competition and the formation of teams have been observed to promote online interaction between participants. However, not all projects have this feature, and it would be of interest to explore whether the introduction of these features had an impact on interaction in a project, or influence how participants contribute to a project,

323 and their motivation to sustain participation. Table 9.2 lists some specific projects that could follow on from this research.

Table 9.2 Future research projects

Exploration of the sustainability of a Foldit. Explore the numbers and composition of the project active and core participants over time. How do the project team work to sustain numbers of participants? Is there a turnover of participants? Longitudinal study of participation and Track the participation and contribution of a small contribution number of active online citizen science participants (either from one project, or from several) over the course of 6 months – 1 year. What influences participation over time? Investigation of non-active registered Foldit or Planet Hunters. Interviews with individuals participants. who showed an initial interest in the project, but then decided not to become active contributors. Is this related to format of project, task, or absence of key motivations? What constitutes ‘scientific research’ for Explore opinions and views of citizen scientists in citizen scientists? order to ascertain how they view / define scientific research. What activities / processes do they think it encompasses? Where do they ‘fit in’? Further exploration of teams and Planet Hunters. What impact does the introduction of competition in online citizen science these features have on interaction between projects. participants, level and type of contribution, and motivation to participate?

9.4 Final reflections

This thesis has examined key aspects of participation in online citizen science projects, namely, who participates, why they participate, and how they participate. It has considered the views and experiences of both the scientists and developers who set up and manage these projects, and the views and experiences of volunteers, some of whom contribute a significant amount of time and resources. Online citizen science projects offer volunteers an opportunity to become involved in authentic scientific research.

Feedback from surveys and interviews with participants suggest that this opportunity is highly valued. Scientists and developers are able to carry out research with the help of

324 volunteers that may not have been possible otherwise. Significant contributions to knowledge have been made and communicated widely both in the academic literature and in the print and online press. While the numbers of active participants may be smaller than those who initially register, online citizen science projects have great potential to involve non-specialists citizens in the production of new knowledge. These volunteers must be supported, however, and future projects need to ensure communities of citizen scientists are made to feel involved and their efforts appreciated and acknowledged in ways that are meaningful to them.

325

References Adams, A. & Cox, A. L. (2008). Questionnaires, in-depth interviews and focus groups. In:

CAIRNS, P. & COX, A. (eds.) Research Methods for Human Computer Interaction.

Cambridge: Cambridge University Press.

Adams, T. (2012). Galaxy Zoo and the new dawn of citizen science. The Guardian [Online].

Available: http://www.theguardian.com/science/2012/mar/18/galaxy-zoo-

crowdsourcing-citizen-scientists [Accessed 19-3-12].

Abawayj, J. (2012). 'Analysis of asynchronous online discussion forums for collaborative

learning'. International Journal of Education and Learning, 1, pp 11-22.

Alexander, A. (2008). From SETI@home to hominid fossils: citizen cyberscience reshapes

research landscape. The Planetary Society [Online]. Available:

http://www.planetary.org/programs/projects/setiathome/setiathome_20080115.

html [Accessed 24/10/11].

Andersen, E., O'rourke, E., Liu, Y., Snider, R., Lowdermilk, J., Truong, D., Cooper, S. &

Popovic, Z. (2012). 'The impact of tutorials on games of varying complexity'.

Proceedings of the 2012 ACM annual conference on Human Factors in Computing

Systems, 2012 Austin, Texas, USA. 2207687: ACM, 59-68.

Anderson, D. P. (2004). Public computing: reconnecting people to science. Conference on

Shared Knowledge and the Web. Madrid, Spain. Available at:

http://boinc.berkeley.edu/boinc2.pdf

Anderson, J. Q. & Rainie, L. (2012). Gamification: expert expect 'game layers' to expand in

the future, with positive and negative results. Pew Internet & American Life

Project. Washington: Pew Research Center.

326

Arthur, C. (2006). What is the 1% rule? The Guardian [Online], Available:

http://www.theguardian.com/technology/2006/jul/20/guardianweeklytechnology

section2 [Accessed 25/04/14].

Baker, S. E. & Edwards, R. (2012). Introduction. In: BAKER, S. E. & EDWARDS, R. (eds.) How

many qualitative interviews is enough? Expert voices and early career reflections

on sampling and cases in qualitative research. National Centre for Research

Methods Review Paper, pp 3-6.

Batson, C. D., Ahmad, N. & Tsang, J. A. (2002). 'Four Motives for Community Involvement'.

Journal of Social Issues, 58, pp 429-445.

Battaglia, M. (2008). Convenience sampling. In: LAVRAKAS, P. J. (ed.) Encyclopedia of

Survey Research Methods. Sage Research Methods.

Bauer, M. W., Allum, N. & Miller, S. (2007). 'What can we learn from 25 years of PUS

survey research? Liberating and expanding the agenda'. Public Understanding of

Science, 16, pp 79-95.

Baxter, P. & Jack, S. (2008). 'Qualitative case study methodology: study design and

implementation for novice researchers'. The Qualitative Report, 13, pp 544-559.

Beaton, B. (2013). Drawing Crowds to Citizen Science Data Collection and Analysis as

Everyday Gaming. University of Nevada Las Vegas Center for Gaming Research.

Podcast available at: http://digitalscholarship.unlv.edu/gaming_podcasts/2/

[Accessed 3/10/13]

Beberg, A. L., Ensign, D. L., Jayachandran, G., Khaliq, S. & Pande, V. (2009).

Folding@home: lessons from eight years of volunteer distributed computing. IDPS

2009: IEEE Symposium on Parallel and Distributed Computing.

Becker, H. S. (2012). How many qualitative interviews is enough? In: BAKER, S. E. &

EDWARDS, R. (eds.) How many qualitative interviews is enough? Expert voices and 327

early career reflections on sampling and cases in qualitative research. National

Centre for Research Methods Review Paper, P. 15.

Bell, P., Lewenstein, B., Shouse, A. W. & Feder, M. A. (2009). Executive Summary. In: BELL,

P., LEWENSTEIN, B., SHOUSE, A. W. & FEDER, M. A. (eds.) Learning Science in

Informal Environments: People, Places and Pursuits. Washington: The National

Academies Press.

Benkler, Y. (2006). The Wealth of Networks. How social production transforms markets

and freedom, New Haven, Yale University Press.

Benkler, Y. & Nissenbaum, H. (2006). 'Commons-based peer review and virtue'. Journal of

Political Philosophy, 14, pp 394 - 419.

Bertrand, C. & Bourdeau, L. (2010). 'Research interviews by Skype: A new data collection

method'. In: ESTEVES, J., ed. 9th European Conference on Research Methodology

for Business and Management Studies, 2010 Madrid.

Bhattacharkee, Y. (2013). 'Mr. Borucki's lonely road to the light'. Science, 340, pp 542-545.

Birch, H. (2010). 'Podologues: conversations created by science podcasts'. New Media &

Society, 12, pp 889-909.

Blank, G. & Reisdorf, B. C. (2012). 'The participatory web. A user perspective on Web 2.0'.

Information, Communication & Society, 15, pp 537-554.

Bohannon, J. (2005). 'Grassroots supercomputing'. Science, 308, pp 810-13.

Bohannon, J. (2009). 'There's power in a puzzle'. Wired, 17, pp 106 -109.

Bonetta, L. (2009). 'New citizens for the life sciences'. Cell, 138, pp 1043-45.

Bonney, R., Ballard, H., Mccallie, E., Phillips, T., Shirk, J. & Wilderman, C. C. (2009a). Public

Participation in Scientific Research: Defining the Field and Assessing its Potential

for Informal Science Education, A CAISE Enquiry Group Report. Washington DC:

Center for the Advancement of Informal Science Education (CAISE). 328

Bonney, R., Cooper, C. B., Dickinson, J., Kelling, S., Phillips, T., Rosenburg, K. V. & Shirk, J.

(2009b). 'Citizen Science: a developing tool for expanding science knowledge and

scientific literacy'. Bioscience, 59, pp 977 - 84.

Borda, A. & Bowen, J. P. (2009). Virtual collaboration and community. In: WHITWORTH, B.

& DE MOOR, A. (eds.) Handbook of Research on Socio-Technical Design and Social

Networking Systems. Hershey, New York: Information Science Reference.

Borgman, C. L. (2007). Scholarship in the Digital Age, London, MIT Press.

Borrell, J. (2013). '10 Great Citizen Science Projects'. The Biologist, 60. Available: http://www.jamesborrell.com/wp-content/uploads/2013/06/Citi-Sci-.pdf, [Accessed

18/10/13]

Bos, W. & Tarnai, C. (1999). 'Content analysis in empirical social research'. International

Journal of Educational Research, 31, pp 659-671.

Bourzac, K. (2008). Biologist enlist online gamers. Technology Review [Online]. Available:

http://www.technologyreview.com/biomedicine/20738/ [Accessed 17/10/11].

Bowden, R. 2012. Women increase their share of biological science PhDs in United States.

NatureJobs [Online]. Available from:

http://blogs.nature.com/naturejobs/2012/01/13/women-increase-their-share-of-

biological-science-phds-in-united-states [Accessed 13/01/14] .

Bowdley, D. 2009. RE: Female membership of the British Astronomical Association.

Personal communication to CURTIS, V.

Bowman, G. R., Voelz, V. A. & Pande, V. S. (2011). 'Taming the complexity of protein

folding'. Current opinion in structural biology, 21, pp 4-11.

Bradford, B. M. & Israel, G. (2004). Evaluating volunteer motivation for sea turtle

conservation in Florida, University of Florida Cooperative Extension Service,

Institute of Food and Agricultural Sciences. 329

Brake, D. R. (2014). 'Are We All Online Content Creators Now? Web 2.0 and Digital

Divides'. Journal of Computer-Mediated Communication, 19, pp 591-609.

Brandtzaeg, P. B. & Heim, J. (2009). Explaining participation in online communities. In:

WHITWORTH, B. & DE MOOR, A. (eds.) Handbook of Research on Socio-Technnical

Design and Social Networking Systems. Hershey, New York: Information Science

Reference.

Braun, V. & Clarke, V. (2006). 'Using thematic analysis in psychology'. Qualitative Research

in Psychology, 3, pp 77 - 101.

British Psychological Society (2007). Report of the working party on conducting research

on the internet. Guidelines for ethical practice in psychological research online.

British Psychological Society.

Bruckman, A. (2002). Ethical guidelines for research online. [Online]. Available:

http://documents.kenyon.edu/provost/irb_ethics_online.pdf [Accessed 20-06-

13].

Bruckman, A. (2006). 'Teaching students to study online communities ethically'. Journal of

Information Ethics, 15, pp 82-98.

Bryman, A. (2006). 'Integrating quantitative and qualitative research: how is it done?'.

Qualitative Research, 6, pp 97-113.

Bryman, A. (2012). Social Research Methods (4th edition), Oxford, Oxford University Press.

Buchanan, E. A. & Hvizdak, E. E. (2009). 'Online survey tools: ethical and methodological

concerns of human research ethics committees'. Journal of Empirical Research on

Human Research Ethics, 4, pp 37-48.

Budhathoki, N. R. & Haythornthwaite, C. (2013). 'Motivation for open collaboration:

crowd and community models and the case of OpenStreetMap'. American

Behavioral Scientist, 57, pp 548-75. 330

Burgelman, J. C., Osimo, D. & Bogdanowicz, M. (2010). Science 2.0 (change will happen...).

First Monday [Online], 15.

http://uncommonculture.org/ojs/index.php/fm/article/view/2961/2573

[Accessed 06/06/12]

Burke, A. (2011). Games that solve real problems: crowdsourcing biochemistry. Forbes

[Online]. Available: http://www.forbes.com/sites/techonomy/2011/10/27/games-

that-solve-real-problems-crowdsourcing-biochemistry/ [Accessed 22/2/13].

Camp, T. (2012). ''Computing, We Have a Problem…’'. ACM Inroads, 3, pp 34-40.

Cardamone, C., Schawinski, K., Sarzi, M., Bamford, S. P., Bennert, N., Urry, C. M., Lintott,

C., Keel, W. C., Parejko, J., Nichol, R. C., Thomas, D., Andreescu, D., Murray, P.,

Raddick, M. J., Slosar, A., Szalay, A. & Vandenberg, J. (2009). 'Galaxy Zoo Green

Peas: discovery of a class of compact extremely star-forming galaxies'. Monthly

Notices of the Royal Astronomical Society, 399, pp 1191-1205.

Carifo, J. & Perla, R. J. (2007). 'The common misunderstandings, misconceptions,

persistent myths and urban legends about Likert scales and Likert response

formats and their antidotes'. Journal of Social Sciences, 3, pp 106-116.

Carr, J., Whitelegg, E., Holliman, R., Scanlon, E. & Hodgson, B. (2009). Investigating

gendered representations of scientists, technologists, engineers and

mathematicians on UK children's television. In: HOLLIMAN, R., WHITELEGG, E.,

SCANLON, E., SMIDT, S. & THOMAS, J. (eds.) Investigating Science Communication

in the Information Age. Oxford: OUP.

Carroll, D., Rahmlow, C., Psiaki, T. & Wojtaszczyk, G. (2005). Distributed science.

http://boinc-berkeley.edu/distributing-science-final.pdf [Online]. Available:

http://boinc-berkeley.edu/distributing-science-final.pdf [Accessed 17/01/12].

331

Castillo, J. J. (2009). Convenience sampling. [Online]. Available:

http://explorable.com/conveniencesampling [Accessed 13/11/13].

Catlin-Groves, C. L. (2012). 'The citizen science landscape: From volunteers to citizen

sensors and beyond'. International Journal of Zoology, 2012, pp.

Chang, J. & Yang, J. (2009). What drives people to contribute in peer production projects?

A review of the participating motivation of peer producers. In: IEEE (ed.) Third

International Symposium on Intelligent Information Technology Application

Workshops. Nanchang, China.

Choi, C. Q. (2005). 'Processing for Science'. Scientific American, 292, pp 30-30.

Ciffolilli, A. (2003). Phantom authority, self-selected recruitment and retention of

members in virtual communities: The case of Wikipedia. First Monday [Online], 8.

Available:

http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1108/10

28.

Clary, E. G., Snyder, M., Ridge, R. D., Copeland, J., Stukas, A. A., Haugen, J. & Miene, P.

(1998). 'Understanding and assessing the motivations of volunteers: A functional

approach'. Journal of Personality and Social Psychology, 74, pp 1516-30.

Clavin, W. (2013). Managing the Deluge of 'Big Data' From Space. NASA Jet Propulsion

Labratory [Online]. Available:

http://www.jpl.nasa.gov/news/news.php?release=2013-299&cid=release_2013-

299&utm_source=iContact&utm_medium=email&utm_campaign=NASAJPL&utm_

content=release+201 [Accessed 18/10/13].

Clow, D. (2013). MOOCs and the funnel of participation. Third Conference on Learning

Analytics and Knowledge. April 8-12, Leuven.

332

Cohn, J. (2008). 'Citizen Science: can volunteers do real research?'. Bioscience, 58, pp 192

- 97.

Colwell, B. (2004). 'The Zen of overclocking'. Computer, 37, pp 9-12.

Conrad, C. C. & Hilchey, K. G. (2011). 'A review of citizen science and community-based

environmental monitoring: issues and opportunities'. Environmental Monitoring

Assessment, 176, pp 273 - 291.

Cooper, C. (2013). Zen in the Art of Citizen Science: Apps for Collective Discovery and the

1 Percent Rule of the Web. Scientific American [Online]. Available from:

http://blogs.scientificamerican.com/guest-blog/2013/09/11/zen-in-the-art-of-

citizen-science-apps-for-collective-discovery-and-the-1-rule-of-the-web/

[Accessed 05/04/14].

Cooper, C. B., Dickinson, J., Kelling, S., Phillips, T., Rosenberg, K. V., Shirk, J. & Bonney, R.

(2009). 'Citizen Science: A Developing Tool for Expanding Science Knowledge and

Scientific Literacy'. Bioscience, 59, pp 977-984.

Cooper, S., Treuille, A., Barbero, J., Leaver-Fay, A., Tuite, K., Khatib, F., Cho Snyder, A.,

Beenen, M., Salesin, D., Baker, D., Popovic, Z. & Players, F. (2010). 'The challenge

of designing scientific discovery games'. Proceedings of the 5th International

Conference on Foundations of Digital Games, 2010, Pacific Grove, CA, USA. 40 -

47.

Cooper, S. (2011). A Framework for Scientific Discovery through Video Games. Doctor of

Philosophy, University of Washington.

Cooper, S., Khatib, F., Makedon, I., Lu, H., Barbero, J., Baker, D., Fogarty, J., Popovic, Z. &

Players, F. (2011). Analysis of social gameplay macros in the Foldit cookbook.

Foundations of Digital Games. Bordeaux, France.

333

Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker,

D., Popovic, Z. & Players, F. (2010). 'Predicting protein structures with a

multiplayer online game'. Nature, 466, pp 756 -60.

Cossins, D. (2013). Games for science: playing scientist. The Scientist [Online]. Available:

http://www.the-scientist.com/?articles.view/articleNo/33715/title/Games-for-

Science/ [Accessed 31-1-13].

Crall, A. W., Jordan, R., Holfelder, K., Newman, G. J., Graham, J. & Waller, D. M. (2013).

'The impacts of an invasive species citizen science training program on participant

attitudes, behavior, and science literacy'. Public Understanding of Science, 22, pp

745-764.

Cranshaw, J. & Kittur, A. (2011). The Polymath Project: Lessons from a successful online

collaboration in mathematics. CHI Vancouver.

Creighton, C. J. (2010). 'Inaugural Genomics Automation Congress and the coming deluge

of sequencing data'. Expert Review Of Molecular Diagnostics, 10, pp 849-851.

Cresswell, J. W. (2009). Research Design: Qualitative, Quantitative and Mixed Methods

Approaches. Sage.

Cronje, R., Rohlinger, S., Crall, A. & Newman, G. (2011). 'Does participation in citizen

science improve scientific literacy? A study to compare assessment methods'.

Applied Environmental Education & Communication, 10, pp 135-145.

Crowston, K. & Fagnot, I. (2008). 'The motivational arc of massive virtual collaboration'.

Proceedings of the IFIP WG 9.5 Working Conference on Virtuality and Society:

Massive Virtual Communities, 1-2 July, 2008, Luneberg, Germany.

Crowston, K. & Prestopnik, N. R. (2013). Motivation and data quality in a citizen science

game: A design science evaluation. Forty-sixth Hawai'i International Conference on

System Sciences (HICSS-46). Wailea, HI. 334

Curtis, V. (2013). 'Contemporary issues in scientific engagement: evaluating the

motivations and expectations of those attending a public astronomy event'.

Communicating Astronomy to the Public, 13, pp 14-18.

Curtis, V. (2014a). 'Online citizen science games: opportunities for the biological sciences'.

Applied & Translational Genomics, DOI: 10.1016/j.atg.2014.07.001.

Curtis, V. (2014b). 'Public Engagement Through the Development of Science-Based

Computer Games: The Wellcome Trust’s “Gamify Your PhD” Initiative'. Science

Communication, 36, pp 379-87.

Czerniewicz, L. (2013). 'The changing digitally-mediated scholarship landscape'. 34th

International Association of Scientific and Technological University Libraries, 2013

Cape Town.

Dawson, E. & Jensen, E. (2011). 'Towards a contextual turn in visitor studies: evaluating

visitor segmentation and identity-related motivations'. Visitor Studies, 14, pp 127-

140.

Debenham, M. (2007). Epistolary interviews on-line: A novel addition to the researchers's

palette. JISC TechDis [Online]. Available:

http://www.jisctechdis.ac.uk/techdis/resources/detail/learnersmatter/Epistolary_

Interviews_On-line [Accessed 9-11-12].

Devictor, V., Whittaker, R. J. & Beltame, C. (2010). 'Beyond scarcity: citizen science

programmes as useful tools for conservation biogeography'. Diversity and

Distributions, 16, pp 354 -62.

Devonshire, I. M. & Hathway, G. J. (2014). 'Overcoming the Barriers to Greater Public

Engagement'. PLoS Biology, 12, pp e1001761.

Dewalt, K. M. & Dewalt, B. R. (2002). Participant Observation. A Guide for Fieldworkers,

Walnut Creek, Ca, AltaMira Press. 335

Dickinson, J. L., Zuckerberg, B. & Bonter, D. N. (2010). 'Citizen science as an ecological

research tool: challenges and benefits'. nnual Review of Ecology, Evolution and

Systematics, 41, pp 149 -72.

Dill, K. A. & Maccallum, J. L. (2012). 'The protein-folding problem, 50 years on'. Science,

338, pp 1042-1046.

Dillenbourg, P. (1999). What do you mean by collaborative learning? In: DILLENBOURG, P.

(ed.) Collaborative-learning: Cognitive and computational approaches. Oxford:

Elsevier.

Dobson, C. M. (2003). 'Protein folding and misfolding'. Nature, 426, pp 884-890.

Dutton, W. H. & Blank, G. (2013). Cultures of the Internet: the Internet in Britain. Oxford

Internet Surveys. Oxford: University of Oxford.

Eiben, C. B., Siegel, J. B., Bale, J. B., Cooper, S., Khatib, F., Shen, B. W., Players, F.,

Stoddard, B. L., Popovic, Z. & Baker, D. (2012). 'Increased diels-alderase activity

through backbone remodelling guided by Foldit players'. Nature Biotechnology,

30, pp 190-2.

Elo, S. & Kyngäs, H. (2008). 'The qualitative content analysis process'. Journal of Advanced

Nursing, 62, pp 107-115.

Entertainment Software Association (2012). 2012 Sales, Demographics and Usage Facts:

Essential Facts About the Computer and Video Game Industry. Entertainment

Software Association (US).

Entradas, M., Miller, S. & Peters, H. P. (2013). 'Preaching to the converted? An analysis of

the UK public for space exploration'. Public Understanding of Science, 22, pp 269-

286.

Escobar, O. (2013). 'Upstream public engagement, downstream policy-making? The Brain

Imaging Dialogue as a community of inquiry'. Science and Public Policy, pp sct073. 336

Estrada, T., Pusecker, K. L., Torres, M. R., Cohoon, J. & Taufer, M. (2013). 'Benchmarking

Gender Differences in Volunteer Computing Projects'. eScience (eScience), 2013

IEEE 9th International Conference on, 2013. IEEE, 342-349.

Eurobarometer (2010). Special Eurobarometer 340: Science and Technology. Brussels:

European Commission.

European Commission (2012). She Figures 2012: Gender in Research and Innovation:

Statistics and Indicators. European Commission Directorate-General for Research

and Innovation. Available: http://ec.europa.eu/research/science-

society/document_library/pdf_06/she-figures-2012_en.pdf. [Accessed 25/10/13]

Evans, C., Abrams, E., Reitsma, R., Roux, K., Salmonsen, L. & Marra, P. P. (2005). 'The

Neighborhood Nestwatch Program: Participant Outcomes of a Citizen-Science

Ecological Research Project'. Conservation Biology, 19, pp 589-594.

Eynon, R., Fry, J. & Schroeder, R. (2008). The ethics of internet research. In: FIELDING, N.

G., LEE, R. M. & BLANK, G. (eds.) The SAGE Handbook of Online Research Methods.

London: Routledge.

Field, H. & Powell, P. (2001). 'Public understanding of science versus public understanding

of research'. Public Understanding of Science, 10, pp 421-426.

Flyvbjerg, B. (2006). 'Five misunderstandings about case-study research'. Qualitative

Inquiry, 12, pp 219-45.

Forte, A. & Bruckman, A. (2005). 'Why do people write for Wikipedia? Incentives to

contribute to open-content publishing'. Proceedings of GROUP 2005, November

6-9 2005 Sanibel Island, Florida.

Gerring, J. (2007). Case Study Research. Principles and Practices, Cambridge, Cambridge

University Press.

Gillham, B. (2000). Case Study Research Methods, London, Continuum. 337

Gillham, B. (2005). Research Interviewing. The Range of Techniques. Maidenhead: Open

University Press.

Goetz, J. P. & Le Compte, M. D. (1984). Ethnography and Qualitative Design in Education

Research, Orlando, FLA, Academic Press.

Gomm, R., Hammersley, M. & Foster, P. (2000). Case study and generalization. In: GOMM,

R., HAMMERSLEY, M. & FOSTER, P. (eds.) Case Study Method. London: Sage.

Grand, A., Bultitude, K., Wilkinson, C. & Winfield, A. (2010). 'Muddy the waters or clearing

the stream? Open science as a communication medium'. 11th International

Conference on Public Communication of Science and Technology, December 6 -

10, 2010 New Delhi.

Grand, A., Wilkinson, C., Bultitude, K. & Winfield, A. F. T. (2012). 'Open science: a new

"trust technology"?'. Science Communication, 34, pp 679-89.

Graneheim, U. H. & Lundman, B. (2004). 'Qualitative content analysis in nursing research:

concepts, procedures and measures to achieve trustworthiness'. Nurse education

today, 24, pp 105-112.

Green, T. (2012). Women over 30 are the most enthusiastic mobile gamers. Mobile

Entertainment [Online]. Available: http://www.mobile-ent.biz/industry/market-

data/women-over-30-are-the-most-enthusiastic-mobile-gamers/036312

[Accessed 10/4/14].

Grey, F. (2009). Viewpoint: the age of citizen cyberscience. CERN Courier [Online].

Available: http://cerncourier.com/cws/article/cern/38718 [Accessed 14/11/11].

Grey, F. (2011). Citizen cyberscience: the new age of the amateur. CERN Courier [Online].

Available: http://cerncourier.com/cws/article/cern/46877 [Accessed 5-12-2011].

338

Gross, D. (2012). Turning gamers into citizen scientists. CNN Online. Available:

http://edition.cnn.com/interactive/2012/08/tech/gaming.series/research.html

[Accessed 17/08/12].

Guest, G., Bunce, A. & Johnson, L. (2006). 'How many interviews are enough? An

experiment with data saturation and variability'. Field methods, 18, pp 59-82.

Guest, G. (2013). 'Describing Mixed Methods Research An Alternative to Typologies'.

Journal of Mixed Methods Research, 7, pp 141-151.

Guest, G., Macqueen, K. M. & Namey, E. E. (2012). Applied thematic analysis, Sage.

Gura, T. (2013). 'Citizen science: Amateur experts'. Nature, 496, pp 259-61.

Gurian, E. H. (2005). Threshold fear. In: MACLEOD, S. (ed.) Reshaping Museum Space.

Chapter 15, pp 203-13. Oxford: Routledge.

Haklay, M. (2011a). Citizen science as participatory science. Po Ve Sham [Online Blog].

Available from: http://povesham.wordpress.com/2011/11/27/citizen-science-as-

participatory-science/ [Accessed 28/11/11].

Haklay, M. (2011b). Classification of citizen science activities. Po Ve Sham [Online Blog].

Available from: http://povesham.wordpress.com/2011/07/20/classification-of-

citizen-science-activities/ [Accessed 12/10/11].

Hammersley, M. & Atkinson, P. A. (2007). Ethnography: prinicples in practice (3rd edition),

Sage.

Hammersley, M. & Gomm, R. (2000). Introduction. In: GOMM, R. H., M. & FOSTER, P.

(eds.) Case Study Method. London: Sage.

Hand, E. (2010). 'People Power'. Nature, 466, pp 685-87.

Hanna, P. (2012). 'Using internet technologies (such as Skype) as a research medium: a

research note'. Qualitative Research, 12, pp 239-42.

339

Hanson-Smith, E. (2013). Online Communities of Practice. In: CHAPELLE, C. A. (ed.) The

Encyclopedia of Applied Linguistics. First ed.: Blackwell Publishing Ltd.

Hars, A. & Shaosong, O. (2002). 'Working for free? Motivations for participating in open-

source projects'. International Journal of Electronic Commerce, 6, pp 25-39.

Haythornthwaite, C. (2009). 'Crowds and communities: Light and heavyweight models of

peer production'. Hawaii International Conference On Systems Science, 2009 Big

Island, Hawaii. IEEE.

Helsper, E. J. (2010). 'Gendered internet use across generations and life stages'.

Communication research, 37, pp 352-374.

Hertel, G., Niedner, S. & Herrmann, S. (2003). 'Motivation of software developers in Open

Source projects: an internet-based survey of contributors to the Linux kernel'.

Research Policy, 32, pp 1159-77.

Hey, T. & Trefethen, A. (2008). E-science, cyberinfrastructure, and scholarly

communication. In: OLSON, G. M., ZIMMERMAN, A. & BOS, N. (eds.) Scientific

Collaboration on the Internet. Cambridge, MA: MIT Press.

Hodson, H. (2013). 'Gamers unleash swarms of nanoparticles on tumours'. New Scientist,

219, pp 21.

Holliman, R. (2005). 'Reception analyses of science news: Evaluating focus groups as a

method'. Sociologia e Ricerca Sociale, 76-77, pp 254-264.

Holliman, R. (2010). 'From analogue to digital scholarship: implications for science

communication researchers'. Journal of Science Communication, 9, pp 1 - 6.

Holliman, R. & Curtis, V. (2014). Online Media. In: GUNSTONE, R. (ed.) Encyclopedia of

Science Education. Springer Reference (in press).

Holliman, R. & Jensen, E. (2009). (in)authentic sciences and (im)partial publics:

(re)constructing the science outreach and public engagement agenda. In: 340

HOLLIMAN, R., WHITELEGG, E., SCANLON, E., SMIDT, S. & THOMAS, J. (eds.)

Investigating Science Communication in the Information Age. Oxford: Oxford

University Press.

Holliman, R. & Scanlon, E. (2006). 'Investigating cooperation and collaboration in near

synchronous computer mediated conferences'. Computers & Education, 46, pp

322-335.

Holohan, A. & Garg, A. (2005). 'Collaboration online: the example of distributed

computing'. Journal of Computer-Mediated Communication, 10, pp Article 16.

Holstein, J. A. & Gubrium, J. F. (2003). Inside interviewing: new lenses, new concerns. In:

HOLSTEIN, J. A. & GUBRIUM, J. F. (eds.) Inside Interviewing: New Lenses, New

Concerns. Thousand Oaks: Sage.

Houle, B. J., Sagarin, B. J. & Kaplan, M. F. (2005). 'A Functional Approach to Volunteerism:

Do Volunteer Motives Predict Task Preference?'. Basic & Applied Social

Psychology, 27, pp 337-344.

Howard, E., Aschen, H. & Davis, A. K. (2010). Citizen science observations of Monarch

butterfly overwintering in the southern United States. Psyche [Online]. Available:

http://www.hindawi.com/journals/psyche/2010/689301/cta/ [Accessed 1/12/11].

Howard Eckland, E. (2013). Why scientists think there are more women in biology than

physics. Gender & Society [Online]. Available from:

http://gendersociety.wordpress.com/2013/09/13/why-scientists-think-there-are-

more-women-in-biology-than-physics/ [Accessed 17/04/14].

Hudson, J. M. & Bruckman, A. (2004). '“Go away”: participant objections to being studied

and the ethics of chatroom research'. The Information Society, 20, pp 127-139.

Hunter, P. (2003). 'Protein Folding: Theory Meets Disease'. The Scientist, September 8, pp.

341

Hustinx, L., Cnaan, R. A. & Handy, F. (2010). 'Navigating Theories of Volunteering: A

Hybrid Map for a Complex Phenomenon'. Journal for the Theory of Social

Behaviour, 40, pp 410-434.

Iacono, J., Brown, A. & Holtham, C. (2009). 'Research methods - a case example of

participant observation'. The Electronic Journal of Business Research Methods, 7,

pp 39-46.

Iacovides, I., Jennett, C., Cornish-Trestrail, C. & Cox, A. L. (2013). Do games attract or

sustain engagement in citizen science? A study of volunteer motivations. CHI

2013. Paris: ACM.

Internet Advertising Bureau (2011). Gaming Britain: A Nation United By Digital Play.

Internet Advertising Bureau UK.

http://www.iabuk.net/sites/default/files/research-docs/Gaming%20Britain%20-

%20booklet.pdf [Accessed 17/09/12].

Ipsos-Mori (2011). Public Attitudes to Science. United Kingdom: Ipsos MORI; Department

for Business, Innovation and Skills.

Irwin, A. (1995). Citizen Science. A Study of People, Expertise and Sustainable

Development, Routledge.

Irwin, A. (2008). Risk, science and public communication. Third-order thinking about

scientific culture. In: BUCCHI, M. & TRENCH, B. (eds.) Handbook of public

communication of science and technology. London: Routledge.

Irwin, A. & Wynne, B. (1996). Introduction. In: IRWIN, A. & WYNNE, B. (eds.)

Misunderstanding science? The public reconstruction of science and technology.

Cambridge: Cambridge University Press.

Ivie, R. & Tesfaye, C. L. (2012). 'Women in physics'. Phys. Today, 65, pp 47.

342

James, N. & Busher, H. (2009a). The displacement of time and space in online research.

Sage Research Methods.

James, N. & Busher, H. (2009b). Engaging with research participants online. Online

Interviewing. Sage research Methods.

Jamieson, S. (2004). 'Likert scales: how to (ab)use them'. Medical Education, 38, pp 1212-

1218.

Jennett, C., Blandford, A., Brohan, P. & Cox, A. (2014). Designing for Dabblers and

Deterring Drop-Outs in Citizen Science. CHI. Toronto.

Jennett, C., Kloetzer, L., Gold, M. & Cox, A. L. (2013). Sociability in virtual citizen science.

(Proceedings) CHI 2013, 'Designing and Evaluating Sociability in Video Games'

Workshop. Paris. PP 29-32

Jensen, E. & Holliman, R. (2009). Investgating science communication to inform science

outreach and public engagement. In: HOLLIMAN, R., WHITELEGG, E., SCANLON, E.,

SMIDT, S. & THOMAS, J. (eds.) Investigating Science Communication in the

Information Age. Oxford: OUP.

Johnson, R. B. & Onwuegbuzie, A. J. (2004). 'Mixed methods research: a research

paradigm whose time has come'. Educational Researcher, 33, pp 14-26.

Joiner, R., Gavin, J., Brosnan, M., Cromby, J., Gregory, H., Guiller, J., Mara, P. & Moon, A.

(2012). 'Gender, internet experience, internet identification, and internet anxiety:

A ten-year follow up'. CyberPsychology, Behavior and Social Networking, 15, pp

370-372.

Jones, S., Johnson-Yale, C., Millermaier, S. & Pérez, F. S. (2009). 'US college students’

Internet use: Race, gender and digital divides'. Journal of Computer-Mediated

Communication, 14, pp 244-264.

343

Jowett, A., Peel, E. & Shaw, R. (2011). 'Online interviewing in psychology: Reflections on

the process'. Qualitative Research in Psychology, 8, pp 354-69.

Kanefsky, B., Barlow, N. G. & Gulick, V. C. (2001). 'Can distributed volunteers accomplish

massive data analysis tasks?'. 32nd Annual Lunar and Planetary Science

Conference, March 12 - 16, 2001 Houston, Texas.

Kang, I., Lee, K. C., Lee, S. & Choi, J. (2007). 'Investigation of online community behavior

using cognitive map'. Computers in Human Behavior, 23, pp 111-26.

Kawrykow, A., Roumanis, G., Kam, A., Kwak, D., Leung, C., Wu, C., Zarour, E., Players, P.,

Sarmenta, L., Blanchette, M. & Waldispuhl, J. (2012). 'Phylo: A citizen science

approach for improving multiple sequence alignment'. PloS ONE, 7.

Kawulich, B. B. (2005). 'Participant observation as a data collection method'. Forum

Qualitative Sozialforschung/Forum: Qualitative Social Research, 6, pp Article 43.

Keel, W. (2010). 'Welcome to Galaxy Zoo'. Astronomy, 38, pp 30 - 35.

Khatib, F., Cooper, S., Tyka, M. D., Kefan, X., Makedon, I., Popovic, Z., Baker, D. & Players,

F. (2011a). Algorithm discovery by protein folding game players. Proceedings of

the National Academy of Science - Early Edition [Online], 108. Available:

www.pnas.org/cgi/doi/10.1073/pnas.1115898108 [Accessed 7/11/11].

Khatib, F., Dimaio, F., Group, F. C., Group, F. V. C., Cooper, S., Kaznierczyk, M., Gilski, M.,

Krzwda, S., Zabranska, H., Pichova, I., Thompson, J., Popovic, Z., Jaskolski, M. &

Baker, D. (2011b). 'Crystal structure of a monomeric retroviral protease solved by

protein folding game players'. Nature Structural & Molecular Biology, 18, pp 1175

- 77.

King, K. N. & Lynch, C. V. (1998). 'The motivation of volunteers in the nature conservancy -

Ohio Chapter, a non-profit environmental organization'. The Journal of Volunteer

Administration, 16, pp 5-11. 344

Kittur, A., Chi, E., Pendleton, B. A., Suh, B. & Mytkowicz, T. (2007). 'Power of the few vs.

wisdom of the crowd: Wikipedia and the rise of the bourgeoisie'. World Wide

Web, 1(2), pp 1-19.

Klandermans, B. (2003). Collective political action. In: SEARS, D. O., HUDDY, L. & JERVIS, R.

(eds.) Oxford Handbook of Poltical Psychology. Oxford: Oxford University Press.

Klandermans, B. & Oegema, D. (1987). 'Potentials, networks, motivations, and barriers:

Steps towards participation in social movements'. American sociological review,

pp 519-531.

Klawe, M., Whitney, T. & Simard, C. (2009). 'Women in computing - take 2'.

Communications of the ACM, 52, pp 68-76.

Kloetzer, L. (2013). Investigating learning in citizen science. Extreme Citizen Science Blog

[Online]. Available from:

http://uclexcites.wordpress.com/2013/09/25/investigating-learning-in-citizen-

science1-by-dr-laure-kloetzer/ [Accessed 04/11/13].

Könneker, C. & Lugger, B. (2013). 'Public Science 2.0—Back to the Future'. Science, 342,

pp 49-50.

Kostadinova, I. (2011). 'Citizen science - the new helping hand for scientists'. Current

Science, 100, pp 973 - 76.

Kouper, I. (2010). 'Science blogs and public engagement with science: practices,

challenges and opportunities'. Journal of Science Communication, 9, pp.

Kozinets, R. V. (2010). Netnography. Doing Ethnographic Research Online., London, Sage.

Krebs, V. (2010). 'Motivations of cybervolunteers in an applied distributed computing

environment: MalariaControl.net as an example'. First Monday, 15.

http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2783/24

52 [Accessed 08/03/13] 345

Kreiss, D., Finn, M. & Turner, F. (2011). 'The limits of peer production: Some reminders

from Max Weber for the network society'. New Media & Society, 13, pp 243-59.

Kuznetsov, S. (2006). 'Motivations of contributors to Wikipedia'. SIGCAS Computers and

Society, 36, pp.

Labaree, R. V. (2002). 'The risk of 'going observationalist': negotiating the hidden

dilemmas of being an insider participant observer'. Qualitative Research, 2, pp 97-

122.

Lai, E. R. (2011). Collaboration: A Literature Review. Pearson Assessments.

http://images.pearsonassessments.com/images/tmrs/Collaboration-Review.pdf.

[Accessed 014/11/13].

Lakhani, K. R. & Wolf, R. G. (2005). Why hackers do what they do: understanding

motivation and effort in free/open source software projects. In: FELLER, J.,

FITZGERALD, B., HISSAM, S. A. & LAKHANI, K. R. (eds.) Perspectives on Free and

Open Source Software. Cambridge, MA: The MIT Press.

Lane, T. J., Shukla, D., Beauchamp, K. A. & Pande, V. S. (2013). 'To milliseconds and

beyond: challenges in the simulation of protein folding'. Current opinion in

structural biology, pp. 58-65.

Larson, S. M., Snow, C. D., Shirts, M. & Pande, V. S. (2009). 'Folding@home and

Genome@home: Using distributed computing to tackle previously intractable

problems in computational biology'. ArXiV, 0901.0866.

Lave, J. & Wenger, E. (1991). Situated Learning. Legitimate Peripheral Participation.,

Cambridge, Cambridge University Press.

Lievrouw, L. A. (2010). 'Social media and the production of knowledge: a return to Little

Science?'. Social Epistemology, 24, pp 219 - 37.

346

Lincoln, Y. S. & Guba, E. G. (1986). 'But is it rigorous? Trustworthiness and authenticity in

naturalistic evaluation'. New directions for program evaluation, 1986, pp 73-84.

Lintott, C. (2013). RE: Number of 'Talk' comments from Planet Hunters. Personal

communication to CURTIS, V. 13/09/13.

Lintott, C. (2014). RE: Predominance of male participants in the Zooniverse. Personal

communication to CURTIS, V. 29/01/14.

Lintott, C. J., Schawinski, K., Keel, W., Van Arkel, H., Bennert, N., Edmondson, E., Thomas,

D., Smith, D. J. B., Herbert, P. D., Jarvis, M. J., Virani, S., Andreescu, D., Bamford, S.

P., Land, K., Murray, P., Nichol, R. C., Raddick, M. J., Slosar, A., Szalay, A. S. &

Vandenberg, J. (2009). 'Galaxy Zoo: ‘Hanny's Voorwerp’, a quasar light echo?'.

Monthly Notices of the Royal Astronomical Society, 399, pp 129-140.

Lintott, C. J., Schawinski, K., Slosar, A., Land, K., Bamford, S., Thomas, D., Raddick, M. J.,

Nichol, R. C., Szalay, A. S., Andreescu, D., Murray, P. & Vandenberg, J. (2008).

'Galaxy Zoo: morphologies derived from visual inspection of galaxies from the

Sloan Digital Sky Survey'. Monthly Notices of the Royal Astronomical Society, 389,

pp 1179-1189.

Lintott, C. J., Schwamb, M. E., Barclay, T., Sharzer, C., Fischer, D. A., Brewer, J., Giguere,

M., Lynn, S., Parrish, M. & Batalha, N. (2013). 'Planet Hunters: New Kepler planet

candidates from analysis of quarter 2'. The Astronomical Journal, 145, pp 151.

Luczak-Rösch, M., Tinati, R., Simperl, E., Van Kleek, M., Shadbolt, N. & Simpson, R. (2014).

Why won't aliens talk to us? Content and community dynamics in online citizen

science. Eighth International AAAI Conference on Weblogs and Social Media. Ann

Arbor, MI, pp 1-10.

Makriyanni, E. & De Liddo, A. (2010). "Fairy rings" of participation: the invisible network

influencing participation in online communities. In: DIRCKINCK-HOLMFIELD, L., 347

HODGSON, V., JONES, C., DE LAAT, M., MCCONNELL, D. & RYBERG, T. (eds.)

Proceedings of the 7th International Conference on Networked Learning.

Lancaster: Lancaster University.

Mansell, R. (2012). 'Promoting access to digital knowledge resources: Managing the

commons'. International Journal of the Commons, 7 (2), pp 255-277.

Markham, A. & Buchanan, E. (2012). 'Ethical Decision-Making and Internet Research:

Recommendations from the AoIR Working Committee (Version 2.0)'.

http://www.aoir.org/reports/ethics.pdf

Marks, N. (2013). 'Six ideal types of public engagement with science and technology:

Reflections on capital, legitimacy and models of democracy'. DEMESCI -

Internation Journal of Deliberative Mechanisms in Science, 2, pp 33-61.

Maslow, A. H. (1943). 'A theory of human motivation'. Psychological review, 50, pp 370-

96.

Maxwell, J. A. (1996). Qualitative Research Design, Thousand Oaks, Sage Publications.

Mayer-Schonberger, V. & Cukier, K. (2013). Big Data, London, John Murray.

Mayer, A. (2010). 'Phenology and citizen science'. Bioscience, 60, pp 172 - 5.

Mcdermott, I. E. (2011). The lazy man's guide to helping others. The Searcher. April 2011,

pp 8-14.

Mcfedries, P. (2011). The Coming Data Deluge. IEEE SPECTRUM [Online], 48. Available:

http://ieeexplore.ieee.org.libezproxy.open.ac.uk/stamp/stamp.jsp?tp=&arnumber

=5693066 [Accessed 5/4/12].

Mcgonigal, J. (2008). Engagement Economy: the future of massively scaled collaboration

and participation. Palo Alto, CA: Institute for the Future.

348

Mcgonigal, J. (2011). Be a gamer, save the world. Wall Street Journal [Online]. Available:

http://online.wsj.com/article/SB1000142405274870459070457609246030299088

4.html [Accessed 10-1-12].

Mclure Wasko, M. & Faraj, S. (2005). 'Why should I share? Examining social capital and

knowledge contribution in electronic networks of practice'. MIS Quarterly, 29, pp

35-57.

Mcwhinnie, S. (2011). The Demographics and Research Interests of the UK Astronomy and

Geophysics Communities 2010. London: Royal Astronomical Society.

Meadows, K. A. (2003). 'So you want to do research? Questionnaire design'. British

Journal of Community Nursing, 8, pp 562-570.

Miller-Rushing, A., Primack, R. & Bonney, R. (2012). 'The history of public participation in

ecological research'. Frontiers in Ecology and the Environment, 10, pp 285-90.

Mueller, M., Tippins, D. & Bryan, L. (2012). 'The future of citizen science'. Democracy &

Education, 20, pp 1-12.

Mugar, G., Osterland, C., Devries Hassman, K., Jackson, C. B. & Crowston, K. (2014).

'Planet Hunters and Seafloor Explorer: Legitimate peripheral participation theough

social proxies in online citizen science'. 2014 ACM Conference on Computer

Supported Cooperative Work.

Newman, G., Crall, A., Laituri, M., Graham, J., Stohlgren, T., Moore, J. C., Kodrich, K. &

Holfelder, K. A. (2010). 'Teaching citizen science skills online: implications for

invasive species training programs'. Applied Environmental Education and

Communication, 9, pp 276 - 86.

Newman, G., Graham, J., Crall, A. & Laituri, M. (2011). 'The art and science of multi-scale

citizen science '. Ecological Informatics, 6, pp 217 - 27.

349

Newman, G., Wiggins, A., Crall, A., Graham, E., Newman, S. & Crowston, K. (2012). 'The

future of citizen science: emerging technologies and shifting paradigms'. Frontiers

in Ecology and the Environment, 10, pp 298-304.

Neylon, C. (2011). 'Three stories about the conduct of science: Past, future, and present'.

Journal of Cheminformatics, 3, pp 35-40.

Nielsen, M. (2012). Reinventing Discovery. The New Era of Networked Science, Princeton

University Press.

Nov, O. (2007). 'What motivates Wikipedians?'. Communications of the ACM, 50, pp 60-

64.

Nov, O., Anderson, D. & Arazy, O. (2010a). Volunteer computing: a model of the factors

determining contribution to community-based scientific research. International

World Wide Web Conference. Raleigh, North Carolina, USA.

Nov, O., Arazy, O. & Anderson, D. (2010b). Crowdsourcing for science: understanding and

enhancing SciSourcing contribution. ACM CSCW Workshop on the Changing

Dynamics of Scientific Collaborations. Savannah, GA.

Nov, O., Arazy, O. & Anderson, D. (2011a). Dusting for science: motivation and

participation of digital citizen science volunteers. iConference 2011. Seattle,

Washington.

Nov, O., Arazy, O. & Anderson, D. (2011b). 'Technology-mediated citizen science

participation: a motivational model'. In: INTELLIGENCE, A. F. T. A. O. A., ed. Fifth

International Conference on Weblogs and Social Media, 2011. 248-56.

O'brien, H. L. & Toms, E. G. (2010). 'The development and evaluation of a survey to

measure user engagement'. Journal of the American Society for Information

Science and Technology, 61, pp 50-69.

350

O'connor, H., Madge, C., Shaw, R. & Wellens, J. (2008). Internet-based interviewing. In:

FIELDING, N. G., LEE, R. M. & BLANK, G. (eds.) The SAGE Handbook of Online

Research Methods. London: Routledge.

Onwuegbuzie, A. J. & Leech, N. L. (2005). 'On becoming a pragmatic researcher: The

importance of combining quantitative and qualitative research methodologies'.

International Journal of Social Research Methodology, 8, pp 375-387.

Oreg, S. & Nov, O. (2008). 'Exploring motivations for contributing to open source

initiatives: the roles of contribution context and personal values'. Computers in

Human Behavior, 24, pp 2055-73.

Oswald, A. J., Proto, E. & Sgroi, D. (2009). Happiness and productivity IZA discussion paper

No. 4645. Forschungsinstitut zur Zukunft der Arbeit (Institute for the Study of

Labor)

Ottinger, G. (2010). 'Buckets of resistence: standards and effectiveness of Citizen Science'.

Science, Technology and Human Values, 35, pp 244 - 70.

Pande, V. S. (2003). 'Meeting halfway on the bridge between protein folding theory and

experiment'. Proceedings of the National Academy of Science, 100, pp 3555-56.

Pande, V. (2013). RE: Number of Folding@home participants. Personal communication to

CURTIS, V. 25/04/13

Paulos, E., Honicky, R. & Hooker, B. (2008). Citizen science: Enabling participatory

urbanism. In: FOTH, M. (ed.) Handbook of Research on Urban Informatics. IGI

Global, Ch 28, pp 414-436.

Paulos, E., Kim, S. & Kuznetsov, S. (2011). The rise of the expert amateur: citizen science

and microvolunteerism. In: FOTH, M., FORLANO, L., SATCHELL, C. & GIBBS, M.

(eds.) From Social Butterfly to Engaged Citizen. Cambridge, MA: MIT Press.

351

Paulus, T. M. (2005). 'Collaborative and cooperative approaches to online group work:

The impact of task type'. Distance Education, 26, pp 111-125.

Pell, G. (2005). 'Use and misuse of Likert scales'. Medical Education, 39, pp 970.

Ponciano, L., F., B., Simpson, R. & Smith, A. (2014). 'Volunteers' engagement in human

computation astronomy projects'. Computing in Science and Engineering, 4 Feb.

2014. IEEE computer Society Digital Library. IEEE Computer Society,

http://doi.ieeecomputersociety.org/10.1109/MCSE.2014.4.

Preece, J. & Shneiderman, B. (2009). 'The reader-to-leader framework: motivating

technology-mediated social participation'. AIS Transactions on Human-Computer

Interaction, 1, pp 13-32.

Priest, S. (2009). Reinterpreting the audiences for media messages about science. In:

HOLLIMAN, R., WHITELEGG, E., SCANLON, E., SMIDT, S. & THOMAS, J. (eds.)

Investigating science communication in the information age: Implications for

public engagement and popular media. Oxford: Oxford University Press.

Prestopnik, N. R. & Crowston, K. (2012). Citizen science system assemblages:

understanding the technologies that support crowdsourced science. iConference

'12. Toronto: ACM.

Qadir, Z. (2013). Crowdsourcing the dark matter of biology. International Science Grid This

Week [Online]. Available: www.isgtw.org/feature/crowdsourcing-dark-matter-

biology [Accessed 21/3/13].

Raddick, J. (2013a). RE: Number of 'Talk' comments from Planet Hunters. Personal

communication to CURTIS, V. 16/09/13.

Raddick, J. (2013b). RE: Planet Hunters Research. Personal communication to CURTIS, V.

23/05/13

352

Raddick, J., Bracey, G., Gay, P. L., Lintott, C. J., Murray, P., Schawinski, K., Szalay, A. S. &

Vandenberg, J. (2010). 'Galaxy Zoo: exploring the motivations of citizen science

volunteers'. Astronomy Education Review, 9, DOI: 10.3847/AER2009036.

Raddick, M. J., Bracey, G., Gay, P. L., Lintott, C. J., Cardamone, C., Murray, P., Schawinski,

K., Szalay, A. S. & Vandenberg, J. (2013). 'Galaxy Zoo: Motivations of Citizen

Scientists'. Astronomy Education Review, 12, pp.

Rashid, A. M., Ling, K., Tassone, R. D., Resnick, P., Kraut, R. & Riedl, J. (2006). 'Motivating

participation by displaying the value of contribution'. Conference on Human

Factors in Computing Systems, April 22-27, 2006, Montreal.

RCUK (2008). Public Attitudes to Science 2008: a Guide. Research Councils UK Department

of Innovation, Unversities and Skills.

Reed, J., Raddick, J., Lardner, A. & Carney, K. (2013). 'An exploratory factor analysis of

motivations for participating in Zooniverse, a collection of virtual citizen science

projects'. Proceedings of the 46th Annual Hawaii International Conference on

Systems Sciences, 7-11 January, 2013 Maui, HI.

Riesch, H., Potter, C. & Davies, L. (2013a). 'Combining citizen science and public

engagement: the Open Air Laboratories Programme'. Journal of Science

Communication, 12, pp 1-19.

Riesch, H. & Potter, C. (2013b). 'Citizen science as seen by scientists: Methodological,

epistemological and ethical dimensions'. Public Understanding of Science, 23, pp

107-120.

353

Rotman, D., Preece, J., Hammock, J., Procita, K., Hansen, D., Parr, C., Lewis, D. & Jacobs, D.

(2012). 'Dynamic changes in motivation in collaborative citizen-science projects'.

Proceedings of the ACM Conference on Computer Supported Cooperative Work,

Feb 11-15, 2012, Seattle, Washington, USA.

Rowles, T. A. (2013). 'Power to the people: does Eterna signal the arrival of a new wave of

crowd-sourced projects?'. BMC Biochemistry, 14, pp 26.

Roy, H. E., Pocock, M. J. O., Preston, C. D., Roy, D. B., Savage, J., Tweddle, J. C. & Robinson,

L. D. (2012). Understanding Citizen Science and Environmental Monitoring. Final

Report on behalf of UK-EOF. NERC Centre for Ecology & Hydrology and the Natural

History Museum.

Ryan, G. W. & Bernard, H. R. (2003). 'Techniques to identify themes'. Field methods, 15,

pp 85-109.

Ryan, R. M. & Deci, E. L. (2000). 'Intrinsic and extrinsic motivations: classic definitions and

new directions'. Contemporary Educational Psychology, 25, pp 54-67.

Ryan, R. M. & Deci, E. L. (2009). Promoting Self-Determined School Engagement.

Motivation, Learning and Well-Being. In: WENTZEL, K. R. & WIGFIELD, A. (eds.)

Handbook on Motivation at School. New York: Routledge.

Salem, P. (2008). 'The seven communication reasons organizations do not change'.

Corporate Communications: An International Journal, 13, pp 333-348.

Sansom, C. (2011). 'The power of many'. Nature Biotechnology, 29, pp 201-3.

Sapsford, R. (2007). Survey Research (second edition), London, Sage.

Savage, N. (2012). 'Gaining wisdom from crowds'. Communications of the ACM, 55, pp 13-

15.

Scanlon, E. (2013). 'Scholarship in the digital age: Open educational resources, publication

and public engagement'. British Journal of Educational Technology, 45, pp 12-23. 354

Schawinski, K. (2011). Galaxy Zoo and the wisdom of crowds. Sky and Telescope.

Schofield, J. W. (2007). Increasing the generalizability of qualitative research. In:

HAMMERSLEY, M. (ed.) Educational Research and Evidence-based Practice. Los

Angeles: Sage.

Schroer, J. & Hertel, G. (2009). 'Voluntary engagement in an open web-based

encyclopedia: Wikipedians and why they do it'. Media Psychology, 12, pp 96-120.

Schmitt, J. R., Wang, J., Fischer, D. A., Jek, K. J., Moriarty, J. C., Boyajian, T. S., Schwamb,

M. E., Lintott, C., Lynn, S. & Smith, A. M. (2014). 'Planet Hunters. VI. An

Independent Characterization of KOI-351 and Several Long Period Planet

Candidates from the Kepler Archival Data'. The Astronomical Journal, 148, pp 28.

Schwamb, M. E., Orosz, J. A., Carter, J. A., Welsh, W. F., Fischer, D. A., Torres, G., Howard,

A., W., Crepp, J. R., Keel, W. C. & Lintott, C. J. (2013). 'Planet Hunters: A Transiting

Circumbinary Planet in a Quadruple Star System'. The Astrophysical Journal, 768,

pp 127.

Schwamb, M. (2014a). 15 percent of 1 million. Planet Hunters Blog [Online]. Available

from: http://blog.planethunters.org/2014/02/17/15-percent-of-1-million/

[Accessed 19/02/14].

Schwamb, M. (2014b). RE: Surprise at Planet Hunters' appeal. Personal communication to

to CURTIS, V. 04/02/14.

Schwamb, M. (2014c). 'How you can find an exoplanet'. Sky and Telescope, March, pp 18-

23.

Schwamb, M. E. (2013. The Beginning is the End is the Beginning. Planet Hunters Blog

[Online]. Available from: http://blog.planethunters.org/2013/08/29/the-

beginning-is-the-end-is-the-beginning/ [Accessed 30/08/13].

355

Schwamb, M. E., Lintott, C., Fischer, D., Smith, A., Boyajian, T., Brewer, J., Giguere, M.,

Lynn, S., Parrish, M. & Schawinski, K. (2014). 'Planet Hunters: Kepler by Eye'.

American Astronomical Society Meeting Abstracts, 2014.

Scott, P. (2010). 'Foldit Research Paper's 57,000+ Co-authors'. Communications of the

ACM, 53, pp 15-15.

Shenton, A. K. (2004). 'Strategies for ensuring trustworthiness in qualitative research

projects'. Education for information, 22, pp 63-75.

SETI@Home (2006). 'SETI@home Poll Results'.

http://boinc.berkeley.edu/slides/xerox/polls.html [Accessed 12/11/11].

Shirky, C. (2009). Here Comes Everybody: How change happens when people come

together, Penguin UK.

Shuy, R. W. (2003). In-person versus telephone interviewing. In: HOLSTEIN, J. A. &

GUBRIUM, J. F. (eds.) Inside Interviewing: New Lenses, New Concerns. Thousan

Oaks: Sage.

Silverman, D. (2006). Interpreting Qualitative Data (3rd Edition), London, Sage.

Silvertown, J. (2009). 'A new dawn for citizen science'. Trends in Ecology and Evolution,

24, pp 467 - 71.

Simon, N. (2012). Come on in and make yourself uncomfortable. Museum 2.0 [Online].

Available from: http://museumtwo.blogspot.co.uk/2012/02/come-on-in-and-

make-yourself.html [Accessed 14/04/14].

Simpson, R. (2013a). Galaxy Zoo Quench: A New Kind of Citizen Science. The Zooniverse

Blog [Online]. Available from: http://blog.zooniverse.org/2013/07/19/galaxy-zoo-

quench-a-new-kind-of-citizen-science/ [Accessed 10/03/14].

Simpson, R. (2013b). RE: Planet Hunters percentage of Zooniverse Traffic. Personal

communication to CURTIS, V. 06/11/13. 356

Simpson, R. (2013c). RE: Planet Hunters: average length of visit. Personal communication

to CURTIS, V. 06/11/13.

Simpson, R. (2014a). One Million Volunteers. Zooniverse Blog [Online]. Available from:

http://blog.zooniverse.org/2014/02/14/one-million-volunteers/ [Accessed

20/02/14].

Simpson, R. (2014b). More languages, more science: translating Zooniverse projects.

Zooniverse Blog [Online]. Available from:

http://blog.zooniverse.org/2014/02/19/more-languages-more-science-

translating-zooniverse-projects/ [Accessed 21/02/14].

Sproull, L. (2011). 'Prosocial behaviour on the Net'. Daedalus, 140, pp 140 - 53.

Sterba, S. K. & Foster, E. M. (2008). Self-selected sample. In: LAVRAKAS, P. J. (ed.)

Encyclopedia of Survey Research Methods. Sage Reference.

Stilgoe, J. (2009). Citizen Scientists: Reconnecting Science with Civil Society, Demos

London.

Stodden, V. (2010). 'Open science: policy implications for the evolving phenomenon of

user-led scientific innovation'. Journal of Science Communication, 9, pp A05.

Symonds, J. E. & Gorard, S. (2010). 'Death of mixed methods? Or the rebirth of research

as a craft'. Evaluation & Research in Education, 23, pp 121-136.

Talmy, S. (2011). 'The interview as collaborative achievement: interaction, identity, and

ideology in a speech event'. Applied Linguistics, 32, pp 25-42.

Taylor, S. J. & Boyden, R. (1998). Introduction to Qualitative Research Methods: a

Guidebook and Resource, New York, John Wiley and Sons.

Thomas, G. (2009). How to do Your Research Project. A guide for students in education

and applied social sciences, London, Sage.

Thomas, G. (2011a). How to do Your Case Study, London, Sage. 357

Thomas, G. (2011b). 'A typology for the case study in social science following a review of

definition, discourse, and structure'. Qualitative Inquiry, 17, pp 511-521.

Timmer, J. (2011). Gamers create recipes for protein-folding algorithms. Ars Technica

Science News [Online]. Available:

http://arstechnica.com/science/news/2011/11/gamers-create-recipes-for-

protein-folding-algorithms.ars [Accessed 8-11-11].

Tinati, R., Simperl, E., Luczak-Rösch, M., Van Kleek, M. & Shadbolt, N. (2014). Collective

intelligence in citizen science–a study of performers and talkers. Collective

Intelligence. Boston.

Tourangeau, R., Rips, L. J. & Rasinski, K. (2000). The Psychology of Survey Response,

Cambridge, Cambridge University Press.

Tourish, D. & Robson, P. (2006). 'Sensemaking and the Distortion of Critical Upward

Communication in Organizations'. Journal of Management Studies, 43, pp 711-

730.

Trumbull, D. J., Bonney, R., Bascom, D. & Cabral, A. (2000). 'Thinking scientifically during

participation in a citizen-science project'. Science Education, 84, pp 265-75.

Uchoa, A. P., Esteves, M. G. P. & De Souza, J. M. (2013). 'Mix4Crowds-Toward a

framework to design crowd collaboration with science'. Computer Supported

Cooperative Work in Design (CSCWD), 2013 IEEE 17th International Conference

on, 2013. IEEE, 61-66.

Vetter, J. (2011). 'Introduction: Lay Participation in the History of Scientific Observation'.

Science in Context, 24, pp 127-141.

Wang, J., Fischer, D. A., Barclay, T., Boyajian, T. S., Crepp, J. R., Schwamb, M. E., Lintott, C.,

Jek, K. J., Smith, A. M. & Parrish, M. (2013). 'Planet Hunters. V. A Confirmed

Jupiter-Size Planet in the Habitable Zone and 42 Planet Candidates from the 358

Kepler Archive Data'. The Astrophysical Journal, 776, doi:10.1088/0004-

637X/776/1/10.

Wenger, E. (2006). Communities of practice: a brief introduction. Available:

http://www.ewenger.com/theory/ [Accessed 1-10-12].

Westphal, A., Von Korff, J., Anderson, D., Alexander, A., Betts, B., Brownlee, D.,

Butterworth, A., Craig, N., Gainsforth, Z. & Mendez, B. (2006) Published.

'Stardust@ home: virtual microscope validation and first results'. 37th Annual

Lunar and Planetary Science Conference, March 13-17, League City, Texas,

Abstract number 2225.

Wiederhold, B. K. (2011). 'Citizen Scientists Generate Benefits for Researchers, Educators,

Society, and Themselves'. CyberPsychology, Behavior & Social Networking, 14, pp

703-704.

Wiersma, Y. (2010). 'Birding 2.0: citizen science and effective monitoring in the Web 2.0

world'. Avian Conservation and Ecology, 5, pp.

Wiggins, A. (2010). 'Crowdsourcing science: organizing virtual participation in knowledge

production'. Proceedings of the 16th ACM international conference on Supporting

group work, Nov 7-10 2010 Sannibel Island, Fla.: GROUP2010.

Wiggins, A. & Crowston, K. (2011). 'From conservation to crowdsourcing: a typology of

citizen science'. 44th Hawaii International Conference on System Sciences, 2011.

2764-73.

Wiggins, A. & Crowston, K. (2012). 'Describing Public Participation in Scientific Research'.

iConference, 2012 Toronto. ACM.

Williams, D., Yee, N. & Caplan, S. E. (2008). 'Who plays, how much, and why? Debunking

the stereotypical gamer profile'. Journal of Computer-Mediated Communication,

13, pp 993-1018. 359

Wilson, J. (2000). 'Volunteering'. Annual Review of Sociology, 26, pp 215-240.

Winn, J. & Heeter, C. (2009). 'Gaming, gender, and time: Who makes time to play?'. Sex

Roles, 61, pp 1-13.

World Comminity Grid (2013). Member Study: Findings and Next Steps. World

Community Grid.

http://www.worldcommunitygrid.org/about_us/viewNewsArticle.do?articleId=32

3 [Accessed 01/09/13]

Yaukey, P. H. (2010). 'Citizen science and bird-distribution data: an opportunity for

geographical research'. The Geographical Review, 100, pp 263 - 73.

Yee, N. (2006). 'The labor of fun. How video games blur the boundaries of work and play'.

Games and Culture, 1, pp 68-71.

Yee, N. (2007). 'Motivations of play in online games'. CyberPsychology & Behavior, 9, pp

772-775.

Yin, R. K. (2003). Case Study research. Design and methods, Thousand Oaks, Ca, Sage

Publications Inc.

Zelenski, J. M., Murphy, S. A. & Jenkins, D. A. (2008). 'The happy-productive worker thesis

revisited'. Journal of Happiness Studies, 9, pp 521-537.

Zucker, D. M. (2009). How to Do Case Study Research. School of Nursing Faculty

Publications Series [Online]. Available:

http://scholarworks.umass.edu/cgi/viewcontent.cgi?article=1001&context=nursin

g_faculty_pubs [Accessed 8/3/13].

360

Appendix A: Cover page for Foldit online survey

361

Appendix B: Foldit survey General questions about how you play Foldit

1. What is your Foldit username?

2. Please tell us how you heard about Foldit?

3. How long have you been playing Foldit?

<6 months

6 months -- 1 year

1 Year -- 18 months

18 months -- 2 years

2 -- 3 years

> 3 years

4. How much time on average have you spent playing Foldit per week in the last month?

< 2 hours

2 -- 4 hours

5 -- 10 hours

11 -- 15 hours

15 hours or more

5. Which of the following best describes your 'level' in Foldit?

beginner

intermediate

362

advanced

6. Do you participate in the Foldit online discussions and forums either by posting content, reading content or managing content? Why - or why not?

7. If you answered 'yes' to question 6, how many hours (on average) do you spend participating in online forums and discussions per week?

< 1

1-2

2-3

3-4

5 or more

8. Do you play Foldit:

as an individual

with your family

with friends

in a team

Other (please specify):

9. If you play Foldit in a team, which team do you belong to and why? (Optional)

Your views about Foldit

363

10. Why did you decide to participate in Foldit?

11. What do you like best about playing Foldit?

12. How would you describe Foldit to others?

13. Have you discussed Foldit with your family or friends? What do they think of your participation?

14. What do you think are the benefits of online 'citizen science' projects such as Foldit to you personally, and to society in general?

15. Have you recommended this project to others?

16. How would you describe your interactions with other players within the Foldit project?

364

17. Do you think Foldit players should be rewarded for playing? If yes, then what would be the most appropriate way?

18. If you could be part of the team that manages Foldit, what would you do differently (if anything) -- are there any features you would add or remove?

Participation in related activities

19. Do you regularly play other online games or video games? If yes, please list your 5 favourites.

20. Apart from this project, do you take part in other science-related activities? Please check the ones you have taken part in during the last 12 months. (select all that apply)

stargazing

going to science centres and museums

watching scientific television programmes

listening to radio programmes about science

attending public science lectures

365

reading popular science books

attending science festivals

attending open days at local universities

attending café scientifique

Other (please specify):

21. Have you ever taken part in other 'citizen science' projects apart from Foldit? Please let us know which ones and if you still contribute to these projects.

22. Have you ever participated in other online collaboration projects such as open source software or writing content for Wikipedia? If yes, please specify which one(s) and if you are still an active contributor to these projects.

Participant information

23. Are you:

male

female

24. How old are you?

16 or 17

18 -- 21

22 -- 25

26 -- 30

31 -- 35

36 -- 40

41 - 45

46 -- 50

366

51 -- 55

56 -- 60

61 -- 65

Over 65

25. What country do you live in?

26. What is your job / profession / occupation, or previous occupation if retired or unemployed?

27. What is your highest educational level / qualification? (i.e. high school, UK 'A' levels, GSCE, Scottish Highers, undergraduate degree, postgraduate degree, other professional qualification)

28. Have you studied science or technology before? If so, to what level? (i.e. high school, UK 'A' levels, GCSE, Scottish Highers, undergraduate degree, postgraduate degree, other professional qualification)

29. Would you be willing to take part in any follow-up questionnaires or online interviews to explore some of these issues in more detail? If so, please provide a contact email address below. (Optional)

367

Appendix C: Folding@home survey

General questions about your participation in Folding@home

1. How did you first hear about Folding@home?

2. How would you describe your experience level with Folding@home?

New to Folding (under 6 months)

Folding for 6 months - 1 year

Folding for 1 - 2 years

Long-term folder (over 2 years)

Beta tester

Pande Group member

3. Do you participate in the Folding@home online discussions and forums either by posting content, reading content or managing content? Why - or why not?

4. If you answered 'yes' to question 3, how many hours (on average) do you spend participating in online forums and discussions per week?

< 1

1-2

2-3

3-4

368

5 or more

5. Do you belong to a Folding@home team? Which team do you belong to and why? (Optional)

Your views about Folding@home

6. Why did you decide to participate in Folding@home?

7. What do you like best about the Folding@home project?

8. Have you had any problems installing or running Folding@home? Please briefly outline any issues you have had.

9. How would you describe Folding@home to others?

369

10. Have you discussed Folding@home with your family or friends? What do they think of your participation?

11. What do you think are the benefits of online 'citizen science' projects such as Folding@home to you personally, and to society in general?

12. How would you describe your contribution to Folding@home? Do you feel as though you are involved in scientific research?

13. Have you recommended this project to others?

14. Do you have any interaction with other participants within the project, or with members of the Folding@home scientific team? If yes, how would you describe this interaction?

370

15. Do you think Folding@home participants should be rewarded for taking part? If yes, then what would be the most appropriate way?

16. If you could be part of the team that manages Folding@home, what would you do differently (if anything) -- are there any features you would add or remove?

Participation in related activities

17. Apart from this project, do you take part in other science-related activities? Please check the ones you have taken part in during the last 12 months. (select all that apply)

stargazing / amateur astronomy

going to science centres and museums

watching scientific television programmes

listening to radio programmes about science

attending public science lectures

reading popular science books

reading popular science magazines

reading online science material

attending science festivals

attending open days at local universities

attending café scientifique

None of these

Other (please specify):

371

18. Have you ever taken part in other 'citizen science' projects apart from Folding@home? Please let us know which ones and if you still contribute to these projects.

19. How would you rate your experience level with computers in general?

Beginner (sometimes need help with day-today tasks)

Intermediate (proficient at office apps, the Web etc.)

Advanced (can do a small amount of programming)

Professional (currently / previously employed as software developer)

Participant information

20. Are you:

male

female

21. How old are you?

16 or 17

18 -- 21

22 -- 25

26 -- 30

31 -- 35

36 -- 40

41 - 45

46 -- 50

51 -- 55

56 -- 60

372

61 -- 65

Over 65

22. What country do you live in?

23. What is your job / profession / occupation, or previous occupation if retired or unemployed?

24. What is your highest education level / qualification?

High school (UK GSCE)

UK 'A' levels / BTEC/ Scottish Highers

Junior college (UK HND)

Undergraduate degree

Postgraduate degree

Other (please specify):

25. What is your highest educational level / qualification in a scientific or technological subject?

High school (UK GSCE)

UK 'A' levels / BTEC/ Scottish Highers

Junior college (UK HND)

Undergraduate degree

Postgraduate degree

Other (please specify):

373

26. Would you be willing to take part in any follow-up questionnaires or online interviews to explore some of these issues in more detail? If so, please provide a contact email address below. (Optional)

374

Appendix D: Planet Hunters Survey

How do you participate in Planet Hunters?

1. How did you first hear about Planet Hunters?

Zooniverse website

Recommendation (from friend, family, colleague)

TV programme (e.g. Stargazing Live)

Magazine or news article

Online article or link

Local amateur astronomy group

Other (please specify):

2. How long have you been participating in Planet Hunters?

<3 months

3-6 months

6 months -- 1 year

1 Year -- 18 months

18 months -- 2 years

More than 2 years

3. How much time on average do you spend on Planet Hunters per week?

< 2 hours

2 -- 4 hours

5 -- 7 hours

8 -- 10 hours

11 -- 15 hours

15 hours or more

4. Do you ever participate in the Planet Hunters online discussions and forums?

(select all that apply)

375

I read content

I post content

I am a forum moderator

5. If you do participate on the project forum, how many hours (on average) do you spend there per week?

< 1

1-2

2-3

3-4

5 or more

Your views about the project

6. Why did you decide to try Planet Hunters?

7. What do you like best about the project?

8. Have you discussed Planet Hunters with your family or friends? What do they think of your participation?

376

9. What skills do you think you need to participate in Planet Hunters?

10. How would you describe your interactions with other participants?

11. How would you describe your contribution to Planet Hunters? Do you feel as though you are involved in scientific research?

12. What do you think are the benefits of 'citizen science' projects such as Planet Hunters both to you personally, and to society in general?

13. Do you think those who take part in the project should be rewarded for their participation? If yes, then

377 what would be the most appropriate way?

14. If you could be part of the team that manages Planet Hunters, what would you do differently (if anything) -- are there any features you would add or remove?

Participation in related activities

15. Apart from this project, do you take part in other science-related activities? Please check the ones you have taken part in during the last 12 months. (select all that apply)

stargazing / amateur astronomy

going to science centres and museums

watching scientific television programmes

listening to radio programmes about science

looking at online material about science

reading popular science magazines

attending public science lectures

reading popular science books

attending science festivals

attending open days at local universities

attending café scientifique

none of these

Other (please specify):

16. Have you ever taken part in other ‘citizen science’ projects apart from Planet Hunters? Please let us know which ones and if you still contribute to these projects.

378

Participant information

17. Are you:

male

female

18. How old are you?

16 or 17

18 – 21

22 – 25

26 – 30

31 – 35

36 – 40

41 – 45

46 – 50

51 – 55

56 – 60

61 – 65

Over 65

19. What country do you live in?

20. What is your job / profession / occupation, or previous occupation if retired or unemployed?

379

21. What is your highest educational level / qualification?

High school (UK GSCE)

UK 'A' levels / BTEC/ Scottish Highers

Junior college (UK HND)

Undergraduate degree

Postgraduate degree

Other (please specify):

22. Have you studied science or technology before? If so, to what level?

High school (UK GSCE)

UK 'A' levels / BTEC/ Scottish Highers

Junior college (UK HND)

Undergraduate degree

Postgraduate degree

Other (please specify):

23. Would you be willing to take part in any follow-up questionnaires or online interviews to explore some of these issues in more detail? If so, please provide a contact email address below.

380

Appendix E Interview questions for citizen scientists

Interview Questions for Foldit Players

1. How did you learn to play Foldit? What resources did you use? If you had to start

again would you take the same route? Do you help others to learn?

2. In your experience, what skills and knowledge are essential to be a good Foldit

player?

3. What aspects of Foldit are important to you and why?

4. What have you gained from playing Foldit?

5. What do think are the main functions / roles of the Foldit management team?

What kind of interaction do you have with the scientists who manage the project?

6. Do you feel like you are involved in the process of scientific research? Do you

think you are ‘doing science’ when you are playing Foldit? Do you consider

yourself a scientist?

7. How has your experience of Foldit changed during the time you have been an

active player? How has the game evolved / changed in your opinion?

8. Have you participated in other citizen science projects? How does your

experience with these projects compare with your experience playing Foldit?

9. Are there any other points / issues you would like to raise?

381

Interview questions for Planet Hunters participants

1. Why do you participate in Planet Hunters?

2. What keeps you participating in this project?

3. What connection with astronomy did you have prior to your participation in

Planet Hunters?

4. What connection with science (more generally) did you have prior to your

participation in Planet Hunters?

5. What other online activities or communities do you participate in (e.g. social

media sites, astronomy or science-related forums, software / computing)?

6. How much time do you spend on Planet Hunters in comparison to other hobbies

or interests?

7. What kind of interaction (if any) would you like to have with the project scientists

or with other project participants? What kind of interaction were you expecting

to have through your participation in Planet Hunters?

8. What do you think about the tasks you are asked to complete as part of this

project?

9. What have you learned from your participation in Planet Hunters?

10. Is there anything else you would like to say about your experience with Planet

Hunters?

382

Interview questions for Folding@home participants (Beta Team and overclockers)

1. Why do you participate in Folding@home?

2. What keeps you participating in this project?

3. How long have you been involved in [overclocking / the Beta Team] do you belong

to any particular communities?

4. How do you see this group generally? What do you like about this community?

5. Do you work with others in this group on issues relevant to Folding@home?

6. How do you think this community / group contributes to Folding@home?

7. What have you learned from your participation in Folding@home?

8. Are there any changes you would make to the project?

9. Is there anything else you would like to say about your experience with

Folding@home, or [overclocking / the Beta Team] more generally?

383

Appendix F: Interview questions for scientists and developers

Interview questions for Foldit Team

1. What were your reasons for getting involved with Foldit? What support and

resources were required to get Foldit established?

2. What aspect of Foldit are you involved in? What is required of you to manage

Foldit on a day-to-day basis?

3. What do you think is (or should be) the main role(s) of the Foldit management

team?

4. Why do you think people are attracted to and play Foldit?

5. Foldit is a difficult game to learn, what do you think is the best way to support

new players? What skills do you think Foldit players need?

6. What do you think Foldit players gain from their participation in Foldit?

7. What are the main benefits of working with ‘citizen scientists’? What do they

contribute to the project? What are some of the main challenges?

8. Could you describe the nature of your collaboration with Foldit players?

9. How has Foldit changed or evolved over the time you have been involved with the

project? Do you think a project like Foldit is sustainable over the long-term?

10. Are there any other points / issues you would like to raise?

384

Interview Questions for PH Scientific Team

1. What were your reasons for getting involved with Planet Hunters?

2. Could you describe your role within Planet Hunters? What are the requirements

of this role (e.g. skills / competencies, time commitment)?

3. What do you think is (or should be) the main role(s) of the Scientific Team?

4. Why do you think people are attracted to participate in Planet Hunters?

5. What skills do you think Planet Hunters participants need?

6. What do you think people gain from their participation? (e.g. development of

scientific skills, opportunity for learning, social interaction, opportunities for

collaboration)

7. What have you gained from your involvement in Planet Hunters? What have been

some of the challenges?

8. What kind of opportunities have you had to interact or collaborate with citizen

scientists?

9. Do you take part in any public outreach / public engagement work in addition to

Planet Hunters?

10. Are there any other points / issues you would like to raise?

385

Interview Questions for Folding@Home Project Team

1. What were your reasons for getting involved with (or creating) Folding@home?

2. Could you describe your role within project? What are the requirements of this role (e.g. skills / competencies, time commitment)?

3. What do you think is (or should be) the main role(s) of the Project Team?

4. Why do you think people are attracted to participate in Folding@home?

5. What do you think people gain from their participation? (e.g. skill development, opportunity for learning, social interaction, opportunities for collaboration)

6. What have you gained from your involvement in Folding@home? What have been some of the challenges?

7. What kind of opportunities have you had to interact or collaborate with the participants?

8. Do you take part in any public outreach / public engagement work in addition to Folding@home?

9. Are there any other points / issues you would like to raise?

386