HOW to WRITE a REPRODUCIBLE LAB PROTOCOL Tools for Sharing Experimental Methods Are Abundant

Work / Technology & tools ILLUSTRATION BY THE PROJECT TWINS THE PROJECT BY ILLUSTRATION HOW TO WRITE A REPRODUCIBLE LAB PROTOCOL Tools for sharing experimental methods are abundant. Researchers now need to put them into practice to help others replicate results. By Monya Baker

very laboratory scientist has a horror more than 70% had been unable to replicate a details are crucial. The word ‘standard’ is a red story. The five-minute step they didn’t published experimental result, with incom- flag for this type of thinking, he says. “There is know they needed, which ended up plete detail in the original protocol given as no such thing as a standard protocol: it’s only costing them five months — or five the most common explanation. standard in your lab.” years. Maybe it was swirling the plate That’s no surprise to Tim Errington, director But things are improving, says Elizabeth Iorns, Eas crowded cells were split between culture of research at the Center for Open Science who helped to launch the RP:CB and is chief exec- dishes. Or maybe the published protocol said in Charlottesville, Virginia, who managed utive at Science Exchange in Palo Alto, California, to wash your sample once and heat thrice but the Reproducibility Project: Cancer Biology which provides research-outsourcing services. meant the opposite, so that following the (RP:CB), which launched in 2013 to replicate “There’s a lot of progress in documentation printed instructions destroyed the sample. results of prominent cancer-biology papers. and tools that are available to people to use.” A More than 60% of respondents to a 2016 Errington sees two overarching reasons for move towards automated experiments and data Nature survey said they had tried to repeat poorly explained methods. One is a lack of collection is also helping, she says, and people other scientists’ experiments and been unable incentives or training for better descriptions. are more aware than ever about the need to to do so. A poll of members of the American The other is that researchers assume everyone describe reagents precisely. Society for Cell Biologists similarly found that works the same way and fail to recognize what Such tools will move the needle on

Work / Technology & tools reproducibility only if they are widely used, weeks later than the original experimenters Watertown, Massachusetts; and the Jackson Errington warns. “Even though we have them, had. A flow chart would have eliminated that Laboratory in Bar Harbor, Maine, can act as it’s not systemic yet, or rewarded.” confusion from the start, Errington says. gatekeepers, supplying cell lines, plasmids and Communicating methods more accurately, There are tools that can help, he says. For genetically engineered mice, respectively, that research-improvement advocates say, requires instance, Experimental Design Assistant are quality checked and distributed to other changes to both mindset and workflow. Here from the UK National Centre for the Replace- researchers. Many journals (including Nature) are some strategies that can help. ment, Refinement & Reduction of Animals ask researchers to deposit research materials in Research is a web-based platform to help in the repositories of such institutions. This Document as you go researchers plan animal studies. It captures spares labs the burden of distributing them If nothing else, consider how you’ll share reporting requirements before work begins and ensures that other scientists will receive your experimental methods from day one. and displays them in a clear diagram. However, quality materials in a reasonable time. In the past five years, several publishers of researchers tend to leave out the diagram and Also available are tools to document com- high-profile journals, including Nature, have focus on written paragraphs, he says. mercial reagents in a standardized way. The strengthened requirements for reporting Resource Identification Initiative, for instance, experimental details in their papers. And many “There is no such thing assigns reagents unique identifiers that persist publishers encourage depositing methods in a even if a company goes out of business or modi- repository, such as Nature Portfolio’s Protocol as a standard protocol: fies its catalogue numbers; more than 600 jour- Exchange, protocols.io or Bio-protocol. it’s only standard in nals (including Nature) encourage their use. But waiting to gather these details until your lab.” The identifiers can be used to populate fields manuscript submission will leave researchers in a lab notebook, for example, or exported scrambling to find information that could into in-development manuscripts to ease study have been right at their fingertips, says Lenny Errington urges researchers to think less write-up, troubleshooting and reproducibility. Teytelman, chief executive at protocols.io in about lengthy protocols and more about One often-overlooked strategy for enhanc- Berkeley, California. Logging information as incorporating online tools while they plan and ing reproducibility is to understand the the experiments are done, he says, “makes conduct experiments, so that the information rationale behind every step, says Polyak. That your life easier, rather than harder”. can be shared. (A bonus, he says, is that this can way, researchers can troubleshoot situations show what didn’t work, such as commercial that might be unique to their lab. Off-the-shelf Write less, show more reagents that failed to perform — information kits undermine this ability to reason, she says. In 2014, Kornelia Polyak, a breast-cancer that is otherwise hard to come by.) Anything When it comes to reproducibility, a good pro- researcher at the Dana-Farber Cancer Institute used to share data can also share methods, he tocol “gives you enough details”, says Luciano in Boston, Massachusetts, showed just how notes: electronic lab notebooks, or reposito- Martelotto, director of the single-cell core facil- tricky sharing methods could be (W. C. Hines ries such as Zenodo, GitHub or Figshare. It is ity at Harvard Medical School in Boston. But a et al. Cell Rep. 6, 779–781; 2014). Her collabora- just a matter of recognizing the need to do so. better protocol, he continues, helps research- tors in California were getting flow cytometry ers understand every step, and why one RNA results that were very different from hers, even Link, template, standardize inhibitor, for instance, is used over another. when they shared samples and reagents and Electronic lab notebooks (ELNs), in particular, discussed their methods extensively. Visiting could be just the ticket for linking to lab pro- Communicate each other’s labs revealed the reason: one tocols, reagent details, related experiments One of the most powerful aspects of emerging group disaggregated tumour tissue by vigor- or other essential details as experiments pro- methods-sharing tools is how they enable ous shaking for 6–8 hours, whereas the other ceed, says Ingo Przesdzing, the ELN programme conversations that share tips and extensions, used gentle rocking for 18–24 hours. When both leader at the Berlin Institute for Health. ELNs says Martelotto. He uses protocols.io to host a teams switched to the same, slower method, can be much more efficient at capturing crucial group, called Single Cell Ninjas, to help other results aligned. Now, Polyak’s lab makes infor- details than conventional written documenta- researchers get single-cell studies working. mal videos that new lab members can watch tion is, he says, and tools to tag whether or not Updating, correcting and interacting about before certain experiments even start. experiments worked can help with both sharing methods is becoming a scientific norm, he says. The Journal of Visualized Experiments, based and troubleshooting. But the benefits come Perhaps because these protocols are seen in Cambridge, Massachusetts, lets researchers only if people are trained properly — otherwise, as less formal than papers, other researchers submit text descriptions of methods for peer he says, they use the ELN like a paper book, are more likely to ask questions, says Benjamin review. If accepted, the journal sends in a film without the advantages of a digital solution. Schwessinger, who studies plant–microbe crew for a day and then edits experiments Electronic laboratory and research note- interactions at the Australian National Uni- into short videos, usually 8–15 minutes long. books can also help to standardize methods and versity in Canberra and helped to launch an Researchers do this both to get credit for record-keeping while providing experimental initiative called Reproducibility for Everyone their work and to persuade others to adopt flexibility, says Alastair Downie, head of infor- to help researchers share methods and data. their methods and expand a field, says Moshe mation technology at the Gurdon Institute in Schwessinger says many more people approach Pritsker, who founded the journal in 2006. Cambridge, UK, who helps lab groups there him about his protocols than about his papers, Timelines and diagrams can also be valuable, implement ELNs. Having appropriate templates even if only to thank him for posting them and says Errington. For one replication study that at the ready makes it much easier to commu- explaining the modifications they’ve made for he coordinated, the original authors, reviewers nicate what to do, he says. “It becomes a huge different organisms. He gets alerts when peo- and replicators consulted intensively using advantage; everyone talks the same language.” ple comment on his protocols at protocols.io, written documents to agree on a replication and also on social media. “It comes naturally for protocol, only to find that mice became too Deposit and document reagents people to interact,” Schwessinger says. “You just sick to complete the experiment (K. Eaton Countless experiments go wrong when a need to build a community around it.” et al. eLife 7, e34364; 2018). It turned out that colleague gives a scientist the wrong cell a vague description of time points meant that line or reagent. Now, organizations such as Monya Baker is a senior Comment editor at the replicators started taking measurements the ATCC in Manassas, Virginia; Addgene in Nature.

Correction This Technology Feature gave the wrong name for the Reproducibility Project: Can- cer Biology.