Project Information Overview, History, and Objectives

Prepared By: Jonathan Romano Last Revised: 04/15/2021 Revision History

Rev. Date Author Revision Description

1 02/28/2020 Jonathan Romano Initial copy

2 02/29/2020 Rhiju Das Copy improvements, historical clarification, improvements to platform development goals

3 03/17/2020 Leonard Copy improvements, move content to Oppenheimer software appendix, additional community section, project note for switch design

4 03/18/2020 Jonathan Romano Copy improvements, add note of upcoming COVID-19 challenge

5 04/14/2021 Sharif Ezzat Copy updates, removed value proposition, updated COVID-19 challenge information

Acknowledgements

This document is based on prior work by and discussions with a number of individuals affiliated with the Eterna project, including Rhiju Das, Benjamin Keep, Do Soon Kim, Leonard Oppenheimer, Jennifer Pearl, and Boris Rudolfs.

1 Table of Contents

Revision History 1

Acknowledgements 1

Table of Contents 2

Introduction 3 Purpose 3 Intended Audience 3

Eterna Project Information 3 Overview 3 Description 3 Notable Achievements 4 Projects 4 Papers 4 Press Coverage 5 History 5 Origins 5 Science 5 Software 6 Players/Community 7 Future Objectives 8 Consortium/Commons Organization 8

Appendix 1: Eterna Software 9 Eterna’s Software Development History 9 Eterna’s Future Software Development 10

2 Introduction

Purpose This document provides information about the Eterna project, how it has developed over time, and plans for how it might be maintained and expanded in the future.

Intended Audience This background document helps educate Eterna staff, media, volunteers, and potential and actual commercial, academic, and funding partners.

Eterna Project Information

Overview

Description Eterna is first and foremost a citizen science game, supporting research in basic and applied research to predict and design synthetic ribonucleic acid (RNA) structures. RNA, the extremely powerful molecule at the heart of many cellular processes, acts as the operating system of life based on how it folds into complex shapes.

RNA research has the potential to lead to breakthroughs in diagnosing and treating dangerous and wide-spread diseases. RNA research may also lead to the development of new technologies, outside of medicine, that are currently impossible through conventional means. However, scientists don’t fully understand RNA’s behavior. Creating synthetic designs to carry out a specific function and predicting how well they will perform when created, is extremely difficult - but the diverse community of Eterna players, representing a wide variety of ages, locations, and backgrounds, have tackled this challenge head on, and have done so with great success. The best players still remain better at designing synthetic RNA and solving RNA secondary structures than the best computer algorithms.

Through the abstraction of a puzzle/logic game, individuals with no biological or even scientific background are able to make meaningful contributions to areas such as fundamental biochemistry, diagnostics, therapeutics, material design, and more.

Players are asked to use RNA’s four nucleotides to design a pattern that will form into a specific shape (or shapes) under various conditions. Over time, these design challenges have become increasingly complex, and reflect a wide variety of mechanics in nature. Initially, players design

3 in a simulated environment for studying and playing with RNA designs that use industry-standard software to predict likely RNA foldings. As they gain experience, players reach the virtual lab and a series of lab challenges. Players create molecular designs for specific biomedical and biotechnology targets. At Stanford University, a partnered wet lab manufactures and tests the designs voted on by the community and provides feedback to the players about how well each design performed. Players then as individuals, and a community, analyze results and use this information to create better designs.

Notable Achievements

Projects ● Fundamental Research in Riboswitch Design - Successfully designed RNAs which change structure based in the presence of external molecules; a necessary precondition for other initiatives to create diagnostics as well as treatment controls and safety mechanisms ● OpenVaccine - A challenge to develop a safe mRNA vaccine optimized for stability and efficacy, as well as future challenges to create new antivirals ● OpenTB - Designing a fast, low-cost, easily-deployable diagnostic to help identify early-stage tuberculosis, one of the world’s largest public health crises ● OpenCRISPR - Improving safety of the CRISPR gene editing technology by allowing it to be turned on and off on demand ● Ribosome Challenge - Stabilizing the ribosome to allow for further work in ribosome engineering, including the ability to develop novel materials that may be costly or impossible to produce by other methods

Papers ● RNA design rules from a massive open laboratory - The culmination of Eterna’s initial research, creating a novel RNA design algorithm (Eternabot) from player analyses and strategies ● Principles for Predicting RNA Secondary Structure Design Difficulty - Creation of a benchmark for RNA design algorithms, now widely used. First paper in the scientific literature to include videogame players as lead authors ● Evidence of an Unusual Poly(A) RNA Signature Detected by High-throughput Chemical Mapping - Discovery of a previously unrecognized experimental artifact, first paper in the scientific literature to include only videogame players as authors ● An unexpectedly effective Monte Carlo technique for the RNA inverse folding problem - RNA design algorithm developed by an Eterna player, far surpassing existing algorithms developed by expert groups ● SentRNA and EternaBrain - RNA design algorithms developed from Eterna player solutions and move history

4 Press Coverage ● New York Times ● CNN ● Wired ● NOVA ● Wall Street Journal

History

Origins The idea for Eterna was inspired by Foldit, a successful citizen science game centered around protein folding. At RosettaCon 2009, Carnegie Mellon University (CMU) graduate student Jeehyung Lee first presented the idea publicly as a modified version of Foldit that could be used for RNA after discussions in preceding months with his advisor Dr. Adrien Treuille (one of the creators of Foldit), and Dr. Rhiju Das (computational biochemist at Stanford University). They decided they wanted to create a new type of citizen science game that incorporated an online open laboratory into the experience.

The project started taking shape through a software development team led by Treuille and Lee at CMU and an experimental biochemistry lab team at Stanford led by Das. Eterna went into beta testing at the end of 2010, and fully launched in January 2011. Since then, the project has continued to grow and evolve to tackle bigger and bigger challenges.

Science Eterna’s initial experiments were done using SHAPE chemical mapping, verifying whether or not each RNA base was either bound or unbound as desired. Initial puzzles were “single state”, but evolved to also include “switch” puzzles, where the objective is to design an RNA that is able to change shape after the introduction of a small molecule (in early experiments, FMN). The focus of the research was to attempt to deduce principles of how RNA folds and how to create successful designs. Players proved to outperform existing algorithms, and provided their strategies through Eterna’s “Strategy Market'' for the creation of a novel design algorithm, Eternabot. These initial studies were presented in the paper RNA design rules from a massive open laboratory published in PNAS, with Eterna players listed as a consortium author.

In 2013, the Das Laboratory developed new synthesis techniques allowing for high-throughput synthesis and analysis of RNA designs - dubbed “Cloud Lab.” Instead of 8, or at most tens, of designs synthesized per lab round, the Das Laboratory increased their capacity to hundreds or thousands of player designs at a time. The “Cloud Lab” expansion also allowed for something novel - the ability for players to propose their own experiments, which was done throughout the lifetime of the Cloud Lab. These efforts would help to lead to the publication of the first paper

5 with Eterna players as lead authors, Principles for Predicting RNA Secondary Structure Design Difficulty, and a paper with only Eterna players as authors, Evidence of an Unusual Poly(A) RNA Signature Detected by High-throughput Chemical Mapping.

In 2015, the Das Laboratory introduced a new experimental method: the RNA Array (also referred to “imaging stations”). The new method used the binding of fluorescent molecules (the MS2 viral coat protein) to RNA as a readout to determine the effectiveness of riboswitches, RNA molecules whose structures change when they bind other molecules. With the ability to test thousands of players’ designs for RNA switch puzzles at a time, Eterna players began to tackle some of their hardest challenges yet: building RNA computers. And this time, instead of small molecules, players would be attempting to bind oligonucleotides (oligos), small pieces of RNA or DNA. Eterna players quickly demonstrated their ability to design logic gates and ratiometric calculators using RNA binding to multiple oligos.

The testing of multi-state RNAs on the RNA Array was perfect preparation for Eterna’s first medically-relevant challenges, OpenTB, aiming to create a fast, low-cost, easily-deployable diagnostic for tuberculosis, one of the world’s largest public health crises, and OpenCRISPR, aiming to create designs which would improve the safety of the CRISPR gene editing technology by allowing it to be turned on and off on demand. IN 2017, the OpenTB project led to successful RNA calculator molecules, which Stanford will patent with Eterna players as named co-inventors. The Das Laboratory is testing these molecules in real-world diagnostic devices. And in 2018, Eterna returned to more basic research in riboswitch design, exploring designing RNA switches using new types of small molecules.

In 2019, Eterna piloted our next major project, the Ribosome Challenge: Players were challenged to stabilize the ribosome to allow for further work in ribosome engineering, including the ability to develop novel materials that are costly or impossible to produce by other methods. Because this project presented a novel challenge in designing significantly longer strands of RNA, players were given added information during the design to suggest which molecular areas were best to mutate given scientific knowledge of existing ribosomes that evolved in bacteria and how some mutations impact ribosome function, based on experimental results from the lab of collaborator Dr. Michael Jewett (Northwestern University).

Software Eterna has been broadly enabled by inspired software developers and a linked series of software algorithms, platforms and tools. Eterna has grown around a gamified website to support collaboration (chat, messaging, lab challenges and lab and player feedback) and scientific inquiry/puzzle challenges (supported by state-of-the-art and evolving energy models, lab challenges, practice and fun puzzles, & new puzzle types (e.g. switches) to support and explore the evolving science).

The software was initially crafted at CMU, and later at Stanford. Energy models available within Eterna have been created from published research of the academy (e.g. University of Vienna)

6 and from Eterna lab and player results. Developers have included academics working on Eterna as part of their research as well as players acting as volunteers and sometimes turning into part-time developers.

For the history of and more details about the software underneath Eterna, please see “Appendix 1: Eterna Software” at the end of this document.

Players/Community As a crowdsourced citizen-science project, Eterna has successfully attracted the interest and effort of thousands of people who might otherwise have never been involved in scientific research. Eterna players hail from all parts of the world and represent people as young as age 10 and as old as octogenarians. Players are attracted by the idea of a fun, free puzzle-solving “game” that promises to use their efforts to help our world. When they join many new players say variations of “I joined to cure cancer/ebola/coronavirus.” The Eterna players have created and sustained a healthy, civil, friendly and caring on-line community over 8 years. Players have challenged themselves and their peers to bootstrap our project with ideas, RNA designs, software suggestions, algorithms, scripts and bots, and player education and outreach.

Our community has been a pivotal part of Eterna and our progress. Here are some notable events that have shaped our community: - Historically, the community has been centered around Eterna’s embedded chat capability, and its forums where a wide variety of shared analysis is now done. - In 2015, prompted by calls from players, the first conference for Eterna, Eternacon, was envisioned and held. Eternacon has become our annual conference, bringing together top scientists, players, and Eterna developers. - In 2016, a Slack team was started. Initially intended as a communication mechanism to coordinate player-led initiatives in software development (which was difficult to manage purely through the forums and chat), it became an important gathering place for the players most involved in Eterna, and a resource for the Eterna staff to use to get feedback. - In 2019, the players created the Eterna Player’s Alliance (EPA). The EPA, organized by and led by players, is a non-commercial organization formed to represent players’ interests, to organize their efforts, and to promote Eterna and citizen-science. The EPA is represented with a permanent seat at Eterna Commons, the emerging multi-university, multi-organization consortium to promote Eterna science and to help deal with Eterna intellectual property (IP).

7 Future Objectives

Consortium/Commons Organization As Eterna expands to tackle bigger challenges, works with more partners, and creates valuable intellectual property, we have recognized the need to create a formal organizational structure to manage the project. Such an organization needs to be able to handle the following things:

- Holding intellectual property created for or assigned to the Eterna project - Holding intellectual property generated by Eterna players - Filing for intellectual property protection such as patents - Licensing intellectual property to 3rd parties - Raising, collecting, and distributing funds (e.g., funds raised by licenses and donations) - Hiring staff and contractors as needed - Managing project priorities and other administrative decisions - Facilitating partnerships with other organizations

The organizational solution might come in one of two ways: (1) The first would be to set up an independent organization (e.g., a corporation or nonprofit) and either hire or contract the appropriate staff to run it. However, this solution could be complex, time-intensive, and cost-prohibitive. (2) The other option, which could be a stepping-stone to creating an independent organization later, if needed, would set up a structure under the auspices of another organization, such as a university or research non-profit, which would be responsible for handling the details of intellectual property management, holding funds, on behalf of Eterna. At this time, we are trying the latter approach: so we are actively seeking a lead organization to help us manage the Eterna project.

Whatever the organizational decision, the Eterna community now faces a series of issues as we move from basic to applied science and theoretical concerns yield to practical considerations. To facilitate decisions, Eterna will be sponsoring a Stakeholder Engagement Process (SEP) at Eternacon 2020 on July 24th. Eterna does have an early exposure draft of specific policies governing management, project operation, intellectual property, finances, and so forth. The SEP will help inform the next draft of our Eterna policies, especially concerning these three questions: (1) whether Eterna should follow a for-profit or non-profit model, (2) how Eterna joint IP will be assigned among community members and organizations, and (3) whether players will receive any sort of remuneration. An early, simple, and persistent thread of community sentiment has favored a model that any revenue derived from player contributions should be fed back into the Eterna project. But this sentiment is neither final nor comprehensive in handling potential IP and financial outcomes.

8 Appendix 1: Eterna Software

Eterna’s Software Development History 2011: Eterna’s initial release included a Drupal-based website and Flash game.

2012: Eterna 2.0 was released, which included a rewrite and user interface redesign of the website to correspond with the launch of the Cloud Lab.

2013: A host of new features were introduced, including moving to a wiki hosted in-house, a scripting interface to allow players to automate puzzle solving, prototypes for a mobile app and algorithm competition, and access for a few players to be able to make improvements to the website themselves.

2014: A partnership was made with NOVA labs (WGBH, Boston) to create an education module built around Eterna, including updated game graphics and informational content and a prototype for a 3D game interface. This year also saw the departure of Jeehyung Lee, and the arrival of John Nicol, the first developer officially hired onto the staff from the player community. By this point, the contribution from CMU was greatly reduced (with the majority of the graduate team there having completed their involvement). Most development work shifted to a small team of rotating individuals at Stanford.

2015: Many of the improvements from the NOVA Labs module were brought back into Eterna itself. Puzzles with multiple RNA ‘oligo’ strands are introduced.

2016: The “Eterna Medicine” initiative is launched, in preparation for OpenTB. Portions of the website interface are redesigned, the game tutorials are overhauled, a new interface to browse experimental data is developed, and scripts are integrated directly into the game.

2017: A number of smaller incremental improvements are made.

2018: A majority of Eterna’s server infrastructure is overhauled and documented, and the Flash puzzle interface is rewritten to TypeScript.

2019: A number of improvements are made to support new challenges. Initial groundwork was done to prepare for significant refactoring of the puzzle interface, internationalization of the puzzle interface, and a rewritten and redesigned website frontend.

2020: Puzzle interface (TypeScript) source made publically available

9 Eterna’s Future Software Development

There are a number of technical projects in the pipeline for Eterna that would provide significant research, medical, community and/or commercial value, but which have not yet been developed due to funding limitations. We hope these projects would be enabled through funding for a stable development team:

- Resolving ~500 user-submitted tickets including bugs and suggestions for new tools, visualizations, workflow improvements, and other enhancements - Rewritten website frontend and backend in modern web frameworks and refactored puzzle interface to ease future extensions - Developing the puzzle interface into a core RNA design application able to be used as an expert-facing tool or embedded into other applications - Overhauled tutorials to improve onboarding and better customized to current scientific challenges - Improved gamification (including rebalanced reward systems and interface feedback during lab design) - Utilities for users to more easily review and evaluate designs, such as grouping, rating systems, new simulation metrics/scoring, and a pipeline for transforming user analysis into concrete metrics - Improved community/collaboration infrastructure, including functionality to support player-initiated projects, such as comprehensive group functionality and more integrated/centralized/functional discussion tools - Internationalization across the entire Eterna platform, including content translation into top 10 major languages of the internet - Mobile-device performant interface for web game; mobile game apps for iOS and Android - Real-time collaborative design within puzzle interface - Secondary, postsecondary, and public education curricula - Classroom management features - In-game solicitation of crowdfunding for urgent challenges, e.g., RNA medicines for virus outbreaks. - New features in the RNA design interface for visualizing protein partners and small molecules that arise in nearly all recent challenges - 3D RNA design interface incorporating tertiary structure modeling tools developed in Das laboratory - New scientific applications, including design of RNA vaccines for pandemic viruses.

10