Rowles BMC Biochemistry 2013, 14:26 http://www.biomedcentral.com/1471-2091/14/26

EDITORIAL Open Access Power to the people: Does Eterna signal the arrival of a new wave of crowd-sourced projects? Thomas A Rowles

Introduction game, though an entertaining diversion, doesn’t warrant What constitutes a scientist, and do you have to be a this? Well, as alluded to above, game players are discov- professional researcher to perform science? ering previously unknown rules about how the primary These are questions that have unwittingly been asked sequence of RNA dictates its structure. In fact, players in recent years by a number of crowd-sourcing initiatives, quickly began to outperform algorithms that had been and none more so than one of the most recent: Eterna [1]. specially designed by experts for the task, and the rules Following in the footsteps of [2], Eterna is an online that they have discovered are now being incorporated puzzle game that is helping to forward our knowledge of into the next generation of algorithms. Does this mean how the primary sequence of RNA molecules determines that the term ‘player’ is no longer appropriate for those their structure. Indeed, not only is Eterna doing a job that who have attained this level? Should we really be starting one would have thought would be restricted to highly- to refer to them as scientists? In order to investigate this, trained scientific researchers and complexly coded al- we were very interested in talking to some of the players gorithms, but it also seems to be doing it better. to explore their motivation, and what they thought the future of this type of this research might be. We were How does Eterna work? kindly provided with the email addresses of several of Before we proceed, we would encourage readers to go to the top game players by the developers of Eterna, and the Eterna website, as this will provide a further under- sent them a number of questions related to their experience standing of the game than we are able to give here. with the game. An exploration of the responses received is However, to summarise: At the beginning of a typical presented below. Eterna round, players are presented with a string of residues and a target conformation. They are What makes an Eterna player? then challenged to form the string into the target simply To generalise, it would appear that there are two broad by changing to other and thereby types of player, though most will obviously sit somewhere taking advantage of the interactions between them. It between these two extremes. These two types can be sounds simple, but it’s not long before you have to start tak- neatly summed up by the response of one player when ing account of how the placement of a particular asked what makes a good Eterna player: will affect its neighbour, as well as the effects of the orienta- “ tion of each pair. In preparing this article we have only There are different profiles of players in Eterna. Some played through some of the simpler levels of the game, but may be only concerned by their ranking, and then, only even there the sense of achievement at coaxing a string into their ability to solve puzzles will count towards defining the target shape is palpable, and at this level the game how good they are. Then, there are those like me who are ” hasn’t even played its trump card. Achieve 10,000 in this for the science and the research. (Nando) points and you earn the right to submit sequences to be synthesised in the laboratory. The developers then feed- In terms of attributes shared by Eterna enthusiasts, back on the accuracy of solutions. Further, at 20,000 points while agreeing that there is a broad spectrum of types of the players earn the right to design their own puzzles. players, the following two responders summed up a Why would the developers go to the trouble of number of useful qualities: synthesising the designs of players though? Surely the “There are all kinds of players, but most of the top Correspondence: [email protected] 10–20 players have 1) an analytical mind, 2) a London, UK

© 2013 Rowles; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Rowles BMC Biochemistry 2013, 14:26 Page 2 of 4 http://www.biomedcentral.com/1471-2091/14/26

methodical approach to , 3) patience “I think that the ability to see patterns, and come up to work through puzzles that take a lot of time to with strategies, helps to make a good player. The fact solve, 4) lots of time on their hands. (Robert Jensen) that non-scientifically trained, but somewhat experienced players, can beat PhD micro-biology “I think anybody can be a good Eterna player, but it students who have not had much practice with the helps to be competitive, curious and have an addictive game in designing in vitro RNA sequences, shows that personality.” (Kevin Cabral) lack of formal training is definitely not a barrier to entry, and formal training is not a guarantee of If we, rather unscientifically, assume that these two success.” (Jeff Anderson-Lee) responses accurately describe the personalities of typical high-level game players, then I think it would be difficult One of the players interviewed also put forward a quote, to argue that there are not marked similarities between made earlier this year by a different player, that attempted the players and a significant number of research scientists. to provide some explanation of why this might be:

Discovering Eterna “The difference between us and scientists (and there The first encounter with Eterna for many players seems are scientists among us), is we have no idea what to have come through the popular media, with both a should work so we try everything. We’re more creative.” Wired magazine article and the PBS television programme (WaterontheMoon) Nova scienceNOW receiving a number of mentions, as did various online science blogs, perhaps belying an underlying Is there a role for Eterna in research? general interest in science. However, what seemed to draw Moving away from the game players specifically, we were many of the players in was the simple appeal of puzzle obviously intrigued by the Eterna initiative itself and what solving. The lure of RNA creation and scientific discovery the future could hold for it. While it is clearly already seems to take hold later. playing a role in research, as it furthers our knowledge of how RNA folds, we wanted to know whether the players “I like solving RNA puzzles, but I would be almost as thought that crowd-sourcing might be the road down happy playing Bejeweled. What makes me come back which science will travel in the coming years. Interestingly, is the lab and the lab results in particular.” (Eli Fisker) despite the obvious respect and praise that they have for the initiative, the players interviewed were refresh- “I have always loved to work puzzles and got hooked ingly pragmatic about its place in the wider world of on these. When I tried the first few puzzles I couldn’t scientific endeavour: solve any of them. They made no sense to me so I was both annoyed and challenged. I slowly figured out how “I think crowd-sourced experiments, like Eterna, will to solve most of them although there are still a couple be a good supplement to what science is already being of puzzle types that I can’t solve. So I keep coming carried out.” (Eli Fisker) back and I will get them in the end!” (Aldwyn Hyatt) “…there is a class of problems in science, requiring to “The initial appeal was the quick ‘success’ of solving an deal with extremely large amounts of data and/or with RNA design puzzle. Later it was the challenge of extremely complex algorithms (those called ‘NP-hard’ by designing RNA to be folded and tested in vitro.” computer scientists), and I do believe that a crowd can (Jeff Anderson-Lee) prove very useful and powerful in those cases.” (Nando)

One of the things that became clear as we read through “Crowd-sourcing has its place in scientific research, the responses is that while none of the players had any but will not, nor should not, replace the individual, higher-level scientific education, the majority did express the ‘drop a flask, pick it up, make discovery’ type of an interest in scientific research, whether this be through research going on in labs around the world.” the popular media or, in one case, through the reading (Anonymous) of scientific papers and their dissemination to other players. Despite this, it was generally felt that a scientific Does Eterna have the potential to become an background did not constitute an advantage when playing educational tool? Eterna. Indeed, as one respondent pointed out, experienced So with an important but perhaps limited role for crowd- non-scientific players generally beat researchers who are sourcing envisaged for research, what did the interviewees new to the game, but may have studied the intricacies of think of the use of Eterna in an educational capacity? molecular biology for years: There is some indication that school groups had been Rowles BMC Biochemistry 2013, 14:26 Page 3 of 4 http://www.biomedcentral.com/1471-2091/14/26

on ‘trips’ into the game, so it appears that this might interested in reading. Like the other players, I post already be happening. informative pictures to promote better understanding within the game, especially for newbies but also for “We have had teachers bring their classes into the advanced players to consider.” (Chris Couteau) game. I think it is a good way to learn, not just about RNA, but also about doing science.” (Eli Fisker) What if player teaching and education weren’t the only un-envisaged factors to come out of Eterna though? What “I’m pretty sure it is a fun way to present RNA to if players began to bring their own unique skills to the high-schoolers. And there’s already been a few ‘visiting party and to start to alter and develop the very nature of classes’ in the past.” (Nando) the game? Fantastically, this is already starting to happen.

In general though, up to this point there did not seem “My greatest achievement is most likely the recent help to have been much consideration that the game could be in getting the scripting interface discussion really used in a formal education capacity such as we had en- moving and bringing software tools sets to their visaged, though some players did see how there was attention to aid the porting of software to JavaScript. potential in this. As a result some of the Vienna RNA software suite has been ported to JavaScript…I have helped establish “I think that as soon as your homework assignment is contacts with the VARNA [3] team to facilitate porting to perform experiments with RNA by playing a video their graphic display software from Java to JavaScript game online, students are not only going to be as well, though the follow through has yet to occur as enthusiastic to learn about science, they will also learn much.” (Chris Couteau) more and perhaps get better grades. Some of these students may even be inspired to become scientists.” “I could say that I have some reasons to feel relatively (Kevin Cabral) proud of having written a software capable of playing and competing with human players in this game What struck us, however, and what we had not envisaged, (the bot is currently ranked 30th).” (Nando) was just how much the game had taught its players, and how much their interest in the underlying concepts There is an argument that the term ‘player’ no longer had been piqued. fits these individuals, but that ‘scientist’ isn’t quite right either. It seems as if they are starting to occupy a grey “Is it a useful teaching tool? Well, ten months ago, area somewhere between player and developer. There is Ididn’t know the first thing about nucleic acids a level of creativity and drive here that Eterna seems to (other than they existed), and if one takes a look at the have drawn out, and which is contributing to the further materials I uploaded on the wiki, I think it could be said development of the game in a way that we imagine the that I’ve been taught quite a few things. Yet, all of it was developers could never have predicted. So how do the my own seeking for more information…Since I started developers feel about this. Is there a concern that the playing and learning about RNA, there hasn’tbeena players are overstepping certain boundaries and tinker- day where I haven’t learned something new about the ing with things that they shouldn’t be playing with? On RNA world.” (Nando) the contrary; it appears that this is a development that designers actively welcome. Indeed, they hold biweekly Interactions between players and community building development meetings with players to discuss thoughts, Even more excitingly than what players had learned ideas and updates to the game. about RNA from the game, we had not envisaged just One of the strongest indications to come out of the how much education and instruction would be going development work described above, and the obvious on between the players. instruction and education that goes on between players, is the strong sense of community that exists between the “…playing Eterna has taught me a little about RNA top game players, and how much of the appeal of playing just from working puzzles, and many players in the it is social, which will of course foster collaboration and community openly share and teach in-game what they creativity, particularly in an environment that appears to know and what they have learned about folding RNA.” be entirely non-judgemental and cooperative. (Lee Bickle) “Most of my success in the game is owed to other “Personally, I maintain on my profile page many links players who have spent much time and effort to papers that the more science orientated may be analyzing, compiling, & documenting what they’ve Rowles BMC Biochemistry 2013, 14:26 Page 4 of 4 http://www.biomedcentral.com/1471-2091/14/26

learned and engaging in dialog with me about the words of Dr Das: “We have been democratizing science for game.” (Lee Bickle) gamers, but now we're democratizing the creation of games for any scientist – expert or citizen!” Conclusions Isthefutureofsciencegoingtobebasedaroundcrowd- To return to our original question, what constitutes a sourced experiments? Our opinion, and that of Eterna scientist, and are the Eterna players challenging this con- players, is no. However, it is a new, potentially powerful tool cept? It could certainly be argued that the players possess in our arsenal that should be considered for ‘big-data’ many of the attributes that define research scientists. They projects. BMC Biochemistry is very excited about seeing are adept at recognising patterns and display formidable what new directions these initiatives take us, and we hope problem solving skills. Perhaps even more importantly, that we will be considering more and more manuscripts they also display a devotion to what they do that some- that integrate results obtained through times seems to border on the obsessive. However, perhaps initiatives with more ‘traditional’ laboratory results in what we’ve really learned in speaking to the players is just the future. how reductive our original question was. In fact, maybe In addition, what we should also take from the Eterna this is where the true innovation of Eterna lies: In realising project is that we should be more creative about developing that the traditional reductionist approach, where we would solutions to scientific problems, and that putting the right think that the only people qualified to undertake research tools into the hands of the right people, even if they are are conventionally trained scientists, is not the only way to not the ones that you might have been expecting, can go about things, and in having the foresight to design the yield incredible results. We like to think that, thanks to tools to take advantage of a previously underutilised work- the open-access nature of BMC Biochemistry,important force. In the end, it doesn’t really matter what attributes sciencewillnotonlybeperformedbymembersofthe Eterna players have, and whether we should consider them public, but that the results of their efforts will also be as scientists or not (for the record, the term favoured by freely available for them to appreciate. the developer Dr Rhiju Das, and which seems to have been ‘ Acknowledgements adopted by the scientific community in general, is citizen Thanks to Rhiju Das and Jeehyung Lee for putting us into contact with the scientists’).Thesimplefactisthattheseareintelligentand Eterna players who were interviewed for this article. Our thanks also to the dedicated people who have been given the opportunity to players for agreeing to be interviewed and who provided us with such open and interesting answers. do something meaningful and worthwhile, and at the same time challenging and fun, and who have grasped this Received: 17 October 2013 Accepted: 17 October 2013 opportunity with both hands. Published: 23 October 2013 ‘ ’ It could be argued that the word foresight above is References misused. After all, the original aim of Eterna was to utilise 1. Eterna. http://eterna.cmu.edu/web/. the so called ‘wisdom of crowds,’ and it could not have been 2. Foldit. http://fold.it/portal/. 3. VARNA. http://varna.lri.fr/. envisaged at inception that it would select for a group of individuals who would have the skills and the inclination to doi:10.1186/1471-2091-14-26 become involved in the future development of the game. Cite this article as: Rowles: Power to the people: Does Eterna signal the arrival of a new wave of crowd-sourced projects?. BMC Biochemistry However, the chance was taken, and the opportunities were 2013 14:26. given to allow the game to develop and evolve in this or- ganic way. In essence, there was a drive and an enthusiasm for exploring the unknown potentials, which perhaps we are tending to see less and less in modern science. The game does not seem to have been developed with a view to the next big publication, nor is granting extra development power to the players going to assist in this. Moreover, it Submit your next manuscript to BioMed Central would appear to have been a simple exploration, both of and take full advantage of: the potential of the tools and of the underlying science of RNA folding, driven by a passion and a joy in what was • Convenient online submission being achieved. Interestingly, for many of the players, these • Thorough peer review seem to be the self-same reasons that they persist in putting • No space constraints or color figure charges so much time and effort into both playing the game, and • Immediate publication on acceptance forwarding its development. Furthermore, this spirit ap- • Inclusion in PubMed, CAS, Scopus and Google Scholar pears to have taken hold with the developers, as they are • Research which is freely available for redistribution now enabling other laboratories to use Eterna for their own Submit your manuscript at crowd-sourced RNA design and modelling projects. In the www.biomedcentral.com/submit