Perspective published: 10 December 2019 doi: 10.3389/fphar.2019.01484

In Vitro : Keeping Up High Standards

Cordula Hirsch 1 and Stefan Schildknecht 2*

1 Particles- Interactions Laboratory, Swiss Federal Laboratories for Materials Science and Technology (Empa), St. Gallen, Switzerland, 2 In vitro and Biomedicine, Department of Biology, University of Konstanz, Konstanz, Germany

Concern regarding the reproducibility of observations in life science research has emerged in recent years, particularly in view of unfavorable experiences with preclinical research. The use of -based systems has increasingly replaced in vivo research and the application of in vitro models enjoys an ever-growing popularity. To avoid repeating past mistakes, high standards of reproducibility and reliability must be established and maintained in the field ofin vitro biomedical research. Detailed guidance documenting the appropriate handling of cells has been authored, but was received with quite disparate perception by different branches in biomedical research. In that regard, we intend to raise awareness of the reproducibility issue among scientists in all branches of contemporary life science research and their individual responsibility in this matter. We have herein compiled a selection of the most susceptible steps of everyday in Edited by: vitro routines that have the potential to influence cell quality and recommend Sebastian Hoffmann, Seh consulting + services, Germany practices to minimize the likelihood of poor cell quality impairing reproducibility with

Reviewed by: modest investment of time and resources. Zhichao Liu, National Center for Toxicological Keywords: cell culture models, reproducibility, toxicology, good cell culture practice, new approach methods (NAM) Research (FDA), United States Albert P. Li, In Vitro ADMET Laboratories, INTRODUCTION United States

*Correspondence: A survey published in Nature in 2016 (Baker, 2016) evaluating questionnaires on reproducibility in Stefan Schildknecht life science research disclosed not only the difficulties researchers have reproducing [email protected] from other laboratories, but also from their own. Even more surprising was the fact that awareness of this problem was widespread within the scientific community. The inability to reproduce study Specialty section: results, often inherent in observations from academic laboratories, are usually uncovered not This article was submitted to without relevant delay, e.g. when potential therapies that are based on these findings transition from Predictive Toxicology, preclinical testing to the far more stringent conditions of clinical trials (Collins and Tabak, 2014). a section of the journal Needless to say, the societal costs associated with this problem are intolerable (Freedman et al., Frontiers in Pharmacology 2015). The controversial matter of insufficient reproducibility was, in fact, communicated openly Received: 21 August 2019 in oncology and cardiovascular biology (Begley and Ellis, 2012; Errington et al., 2014; Libby, 2015). 15 November 2019 Accepted: In toxicology, which may better reflect the background of most readers of this journal, awareness Published: 10 December 2019 of this problem has emerged only gradually in association with insufficientin vivo reproducibility Citation: (Kilkenny et al., 2009; Voelkl et al., 2018). Such disclosures, in concert with studies indicating that Hirsch C and Schildknecht S (2019) In Vitro Research Reproducibility: in vivo data from rats and mice combined can only predict human clinical toxicology of less than Keeping Up High Standards. 50% of candidate pharmaceuticals (Olson et al., 2000), promoted a revision of several toxicologists’ Front. Pharmacol. 10:1484. opinions towards mechanistic in vitro assays from the traditional reliance on pharmacological and doi: 10.3389/fphar.2019.01484 toxicological in vivo .

Frontiers in Pharmacology | www.frontiersin.org 1 December 2019 | Volume 10 | Article 1484 Konstanzer Online-Publikations-System (KOPS) URL: http://nbn-resolving.de/urn:nbn:de:bsz:352-2-10e6snhkxwr5q6 Hirsch and Schildknecht In Vitro Reproducibility Crisis

IN VITRO MODELS IN LIFE SCIENCE particularly in the field of in vitro research. In , RESEARCH in vitro toxicity assays are the most frequently used approaches to assess potential hazardous effects of engineered nanomaterials A major concern raised by researchers in different fields of (Guggenheim et al., 2018). This is mainly due to the fact that biomedicine was how a cell culture model, often not even researchers early on realized that the immense number of newly originating from the organ of interest, could provide information developed nanomaterials would make it impossible to perform about multilayer processes and pathological outcomes in classical in vivo animal tests due to the amount of time, money, humans. In this context, it is important to understand that and number of animals required (Hartung and Sabbioni, 2011; application-oriented fields, such as pharmacology or toxicology, Schrurs and Lison, 2012; Guggenheim et al., 2018). Nanomaterials operate to a large extent on the fundamental progress made exhibit unique properties due to their small size that make them in biomedical research over the past decades and exploit suitable for many different applications. However, these same the wealth of information generated about cellular stress particle properties often interfere with experimental test systems pathways and molecular processes. This paradigm shift was (Wörle-Knirsch et al., 2006; Laurent et al., 2012; Bohmer et al., largely shaped by the US National Research Council’s (NRC) 2018). Insufficient nanoparticle characterization, unidentified strategic plan to modernize methods for testing environmental interference with test systems, and poor definition of controls toxicants (Natl. Res. Counc., 2007). The approach envisions the for monitoring assay performance led to several contradictory identification of molecular targets and pathways that are linked observations in the early days of nanotoxicology research (Hirsch with a toxicological outcome and fosters the establishment and et al., 2011). However, this shaky start allowed nanoparticle validation of high-throughput new approach methods (NAM) toxicology to emerge as one of the fields in which the aspect of for quantitative assessment of target perturbations (Collins et al., adequate reproducibility of in vitro studies gained appropriate 2008; U.S. Environ. Prot. Agency, 2009). A key element in the attention, and consequently, allowed an open discussion of NRC’s strategy is its distinct focus on the quantitative detection this topic. An exemplary illustration of this transparency is a of perturbations of defined molecular events [Key Events, (KE)], publication by Elliott and colleagues (Elliott et al., 2017) assessing cellular stress pathways, and marker signatures that are predictive the reproducibility of MTS-tetrazolium reduction assay results for a specific outcome (Adeleye et al., 2015). The experimental in as indicators of cell viability in an international inter-laboratory vitro model of choice, therefore, needs to express the pathway comparison study with five independent laboratories. Strict or mechanism of interest and also needs to allow quantitative standard operating procedures (SOP) were employed using a determination of the disturbance caused by the stressors. In sophisticated 96-well plate design that allowed detection of up this regard, the concept of adverse outcome pathways (AOP) to seven parameters of assay performance, including accuracy was designed as a conceptual framework for the sequential of multi-channel pipetting, cell handling/cell growth, and organization of the molecular initiating event (MIE), connected instrument performance (i.e. plate reader issues) (Rösslein et al., with the adverse outcome (AO) via a series of KEs (Ankley et al., 2015). A549 cells were purchased from two independent, credible, 2010). The AOP concept fosters the development or selection accepted commercial sources and both, seemingly identical, cell of assays allowing a quantitative detection of individual KEs, cultures were used in all labs. Even under such strict conditions, thereby enabling the definition of threshold levels (Leist et al., EC50 values of the two A549 cultures upon CdSO4 treatment 2017; Terron et al., 2018). AOP also represents an organizational differed by a factor of two in all laboratories. In the course of tool for identification of additive or synergistic effects that might these investigations, cell line authentication was discovered to be occur through activation of identical or different KEs by two or one of the main factors influencing assay results. Short tandem more compounds. The stringent demand for precise quantitative repeat sequencing revealed a partial deletion in one and qualitative information required for AOPs illustrates the of the cell cultures. Technical aspects also contributed to result explicit necessity for experimental models with a high rate of variability. For example, simple cell handling steps, such as PBS reproducibility and the necessity for increased awareness of the washing, were identified to significantly change assay outcomes. reproducibility problem in all branches of life science research. This example provides a vivid illustration of the impact of seemingly trivial details and the necessity to draw attention to all aspects of in vitro experimentation. INSUFFICIENT REPRODUCIBILITY IN A recent evocative study of the mammary epithelial cell CELL MODELS line MCF10A and growth rate inhibition by anti-cancer drugs systematically addressed inter- and intra-study center variations A defined assay performed with a definedin vitro model needs to and identified factors contributing to insufficient reproducibility yield identical results— no matter when or where it is performed. (Niepel et al., 2019). Although the five research centers applied As trivial as this statement may appear, its implementation cells and chemicals of the same stock, astonishing center- is quite difficult in reality. The Nature survey of 2016Baker, ( to-center variations up to 200-fold were observed in growth 2016) highlighted the degree of inadequate reproducibility in inhibition rates. Cell seeding, i.e. slight variations in initial cell biomedical research and underlined the widespread awareness numbers, was identified as one key source of these variations (for of the problem within the scientific community. It is, thus, all the more details see Recommendation 5) (Cell density and medium more astonishing that systematic comparisons of experimental change). Overall, the subtle interplay between experimental models applied in different laboratories are rather rare, methods and a vast array of poorly defined sources of biological

Frontiers in Pharmacology | www.frontiersin.org 2 December 2019 | Volume 10 | Article 1484 Hirsch and Schildknecht In Vitro Reproducibility Crisis variability was found to be the main cause of the observed GOOD CELL CULTURE PRACTICE: irreproducibility. For example, two distinct methods were used SUGGESTIONS TO IMPROVE to quantify cell viability: a) microscopic cell counting as a direct measure of viable cell number and b) detection of intracellular REPRODUCIBILITY ATP levels as a proxy of viable cells. ATP levels do not necessarily To the best of our knowledge, no specific field ofin vitro research directly correlate to the number of viable cells, resulting in faces greater issues of reproducibility than another. No particular identical EC50 values for some drugs, but differing greatly for cell line, cellular model, or particular assay seems to be favorable others. Changes in ATP levels following treatment could be the in this regard. In contrast, all fields ofin vitro toxicology seem consequence of cell death, effects on cell proliferation or the to face certain— though not necessarily the same— issues of alteration of cellular ATP metabolism. Furthermore, linearity irreproducibility. Therefore, the question arises which elements in between cellular ATP levels and cell viability is not justified for in vitro biomedical research are potential sources of unsatisfactory many cell types. In several cases, a reduction of ATP levels by reproducibility, and can be actively influenced by individual almost 50% is tolerated by cells without significant influence researchers with manageable effort and within the framework on cell viability (Pöltl et al., 2012). In conclusion, while both of the existing scientific system. Over the course of the past two assays (direct cell counting and ATP measurements) might decades, initiatives to improve the quality of in vitro research be quite robust and reproducible per se, they provide different have identified critical aspects ofin vitro cell culture routines information from their results, e.g. drugs that alter cellular ATP and their influence on reproducibility. The concept ofGood Cell metabolism, and are thus not interchangeable in these cases. Culture Practice (GCCP) (Coecke et al., 2005) was developed and As a consequence of the huge number of individual biological gradually adapted to ongoing scientific progress (Pamies et al., factors involved, Niepel and colleagues came to the rather 2018) as a guidance document for in vitro reporting standards. discouraging conclusion that “most examples of irreproducibility Other initiatives, such as the OECD guidance document for Good are themselves irreproducible” (Niepel et al., 2019). In Vitro Method Practices (GIVIMP) (Eskes et al., 2017), defined This spectrum of biological factors further depends on the standards for regulatory testing under Good Laboratory Practice complexity of the cell model applied. The introduction of 2D (GLP) rules. Recently, Petersen and colleagues very specifically co-culture models and 3D cell models was motivated by the discussed sources of variability in four distinct nano-bioassays ambition to recapitulate the natural in vivo environment of (Petersen et al., 2019). The following selection of approaches to cells in a cell culture dish. In fact, cells in a 3D culture differ improve reproducibility of in vitro studies was loosely inspired by morphologically and physiologically from their counterparts these initiatives and makes no claim to completeness. It is instead in a 2D setup (Baharvand et al., 2006; Edmondson et al., 2014). based on the experiences of the authors and communications Introduction of the third dimension in a cell culture model with colleagues from adjacent scientific disciplines. An explicit results in additional parameters that could potentially affect emphasis was placed on broadly applicable techniques to reproducibility, including spheroid size and consequently improve the reproducibility of results obtained with cellular in the oxygen and nutrient supplies to cells in different layers vitro models, which can be implemented with relatively little within the structure; spatial organization of surface receptors investment and provide major benefits for the individual project involved in interactions with neighboring cells; activation and the scientific community as a whole.Figure 1 provides a of signal transduction pathways; and induction of gene summary of potential sources of variability that might influence expression profiles (Vinci et al., 2012). All of these changes in vitro results. The particular aspects marked in yellow in the ultimately have the potential to influence cell biology and diagram are discussed in more detail. cellular response towards exogenous stressors. These aspects were exemplified by a study using HCT-116 cells in 3D culture that displayed increased resistance against anti-cancer drugs RECOMMENDATIONS compared with the 2D model (Karlsson et al., 2012). The results in the 3D model more closely reflected in vivo observations. 1. Selection of plasticware: Cell culture flasks and dishes are Nonetheless, the initial euphoria regarding such studies manufactured from different materials, such as became gradually overshadowed by higher rates of insufficient polystyrene or polyethylene terephthalate (PET). Each of these reproducibility observed in complex cell models. To our materials possesses different surface properties that affect the knowledge, no systematic comparison of the reproducibility of interaction of cells with the plastic surface. Even identical cells in mono-culture vs. their integration into more complex formats of cell culture dishes and plates made from the same models (co-culture, 3D, etc.) has yet been published. Rumors type of plastic by different manufacturers can have different from the industry, however, indicate a returning trend towards surface properties as a consequence of production variability more elementary cell models with robust readouts that allow (e.g. temperature, duration of the production process, or both adequate predictivity and high reproducibility. This supplier of raw materials) (Battiston et al., 2012). Surface does not mean that complex cell models are inappropriate parameters, such as hydrophilicity or electrical charge, can with respect to their reproducibility per se, but they require influence attachment and activation states of cells. It has been far higher investment in their characterization and validation shown that monocyte adhesion, cytokine release, maturation to limit the degree of overall variability compared with less of oocytes, and differentiation of neurons, to name just a few, complex models. can be significantly affected by different types of plastic and

Frontiers in Pharmacology | www.frontiersin.org 3 December 2019 | Volume 10 | Article 1484 Hirsch and Schildknecht In Vitro Reproducibility Crisis

FIGURE 1 | Cause-and-effect diagram summarizing potential sources of variability relevant for in vitro assays. According to Petersen et al. (2019) cause-and-effect analysis was applied to visualize sources of variability. We do not claim completeness of the information but rather encourage researchers to complement and/ or adapt the diagram for their specific cellular model or field of research and take the provided information as a starting point to challenge and scrutinize their own working standards. Keywords marked in yellow are discussed specifically in the main text.

different plastic manufacturing processes (Shen and Horbett, PC12 are highly dependent on the coating composition and 2001; Brodbeck et al., 2002; Clinchy et al., 2003; Schildknecht coating procedures applied (Orlowska et al., 2017; Teppola et al., 2009). These examples clearly illustrate the fundamental et al., 2018). Similarly, different coatings reportedly influence importance of an initial characterization of the influence of attachment, nuclei shape, branching of neurites and neuronal the plasticware used for the cell model and the importance of network formation of the neuroblastoma cell line SH-SY5Y. adequately reporting materials. The hepatoma cell line cell line HepG2, which is widely used 2. Coating: Cells embedded within a tissue interact with the not only for liver-associated research, seeded on polystyrene extracellular matrix to develop cell type-dependent features well plates with and without coating exhibited significant such as morphology, function, proliferation, differentiation, differences in their morphology, distribution, and functions gene expression pattern, and even survival (Dike and Farmer, such as particle uptake or P450-dependent detoxification 1988; Longhurst and Jennings, 1998; Damsky and Ilić, 2002). (Saravia and Toca-Herrera, 2009; Prats-Mateu et al., 2014). Likewise, coating of cell culture plastic with extracellular As a large percentage of biomedical research is actually matrix such as collagen, fibronectin, or laminin conducted with such immortal, presumably easy-to-handle offers specific contact partners for interaction with adhesion cell lines, an explicit focus on the precise documentation of receptors (e.g. integrins) on the cell surface (Giancotti and coating details (i.e., coating components, amount, volume Ruoslahti, 1999). Certain cell types strongly depend on an of coating per well, coating period, washing steps, storage appropriate coating and absence of extracellular matrix after coating) is essential to ensure reproducibility. components can even lead to the complete demise of the 3. Cell characterization: A large percentage of research is affected cells. Obviously, in specific fields inof vitro research, conducted with a relatively small number of tumor or e.g. differentiation, the coating receives a great degree immortalized cell lines. Unfortunately, there is a clear trend of attention (Somaiah et al., 2015; Abdal Dayem et al., 2018; that the more unpretentious and widely distributed the cell Sun et al., 2018). This is in contrast to more conventional and line, the less emphasis is spent on its characterization. Long- easy-to-handle tumor or immortalized cell lines, which seem term sub-culturing, for instance, puts selective pressure on to grow on any kind of bare cell culture plastic. However, the individual cells in the culture with higher growth rates. After extracellular matrix has also been shown to influence adhesion, a few passages, this can lead to complete exclusion of slower proliferation, mobility, and morphology of rather robust cell proliferators and may culminate in genetic or phenotypic lines (Fiegel et al., 2004; García-Parra et al., 2013; Liberio et al., alterations, e.g. accumulation of mutations or changes in 2014). For example, in the field of neuroscience, adhesion morphology, development, or gene expression patterns and neurite outgrowth of the pheochromocytoma cell line (Hughes et al., 2007). Systematic investigations revealed

Frontiers in Pharmacology | www.frontiersin.org 4 December 2019 | Volume 10 | Article 1484 Hirsch and Schildknecht In Vitro Reproducibility Crisis

significant genetic differences in widely applied cell models in metabolism and general cell function (Shah et al., 2013). between different laboratories Frattini( et al., 2015; Kleensang Cell culture media in use today inevitably creates conditions et al., 2016; Gutbier et al., 2018). Furthermore, contamination that are not directly comparable to conditions in vivo. An of cell lines with other cells is a frequent problem in in vitro awareness of the constituents of the medium used and their research and can only be excluded by elaborate genetic influence on cellular processes are cornerstones of improved authentication. Additionally, cell lines are often distributed by reproducibility. There is no right or wrong medium or cell informal exchange between laboratories, a process that is far model, but to ensure reproducibility, general cell culture less often accompanied by a transfer of relevant information parameters need to be fixed, and more importantly, the regarding cell passage number or origin of the cell line, let experimenter needs to be aware of the features, strengths and alone genetic characterization. In order to limit these potential weaknesses of the in vitro model when interpreting the data it sources of error, cell lines should be acquired only from generates. established (commercial and non-commercial) cell culture 5. Cell density and medium change: After identification of banks (e.g. ATCC, ECACC, DSMZ, JCRB, CellBank Australia) the so-called “inoculum effect,” which describes increases that guarantee an examination of cross-contamination as well in the minimal inhibitor concentration of an antibiotic as molecular cell authentication on the basis of fingerprinting, with increased numbers of bacterial cells, it soon became single locus short tandem repeats (STR) and single apparent that the density of eukaryotic cells can have a polymorphism (SNP) profiling Capes-Davis( et al., 2010; profound influence on the outcome of toxicological studies Didion et al., 2014). Establishment of a new cell culture from (Ohnuma et al., 1986; Brook, 1989; Schildknecht et al., 2009; a single vial, in general, represents a bottleneck situation with Schildknecht et al., 2011; Scholz et al., 2011). In addition to the potential to select slightly modified "subclones" in the the statistical decrease in drug accumulation (Takemura cell population and hence, deserves particular attention by et al., 1991; Schildknecht et al., 2015), higher cell densities are researchers. Upon arrival and initial thawing of thoroughly characterized by more pronounced paracrine signaling and characterized cells, a large stock of vials (> 100 vials) should contact inhibition. These events can influence cell metabolism, be prepared. This not only guarantees availability of cells with which in turn modulates cellular responses to pharmaceutical a relatively low passage number over years and decades, but or toxicological compounds, and, consequently, cell viability. also allows assessment of the potential influence of freezing/ It is therefore apparent that modest variations in the number thawing by comparison of the initially received cells with cells of seeded cells, e.g. as a consequence of differences in that have undergone a freezing/thawing cycle. counting methods, can lead to significant disparities in cell 4. Selection of cell culture media: Once a cell line is introduced density after a few days of proliferation. High cell densities in a laboratory, little emphasis is often placed on the cell also influence the composition of media constituents and the culture medium. Approximately 90% of all reported in accumulation of waste products, such as lactate or ammonia. vitro experiments are currently conducted with one of four All of these parameters can be manipulated by the frequency different types of medium: DMEM, RPMI, M199; MEM of medium changes (Wright Muelas et al., 2018). Medium (McKee and Komarova, 2017). The fact that the electrolyte changes have been shown to influence oxygen supply and composition and carbohydrate content of these types of cellular metabolism, as well as synthesis of proteins, DNA, medium are usually not ideal physiological environments and RNA (Al-Ani et al., 2018; Wright Muelas et al., 2018). is often overlooked. A typical scenario resulting from the These observations clearly highlight the necessity for use of these media is an oversupply of phosphate and/or a precise documentation and reporting of parameters like cell limitation of calcium and magnesium (McKee and Komarova, density and frequency of medium changes to minimize their 2017). Typical plasma glucose levels are in the range of ca. contribution to insufficient reproducibility. 5 mM in healthy humans, yet many commonly used media 6. contamination: Receipt of (often non-tested) cells contain 25 mM glucose. Such conditions support reliance from other laboratories bears the threat of contamination by on glycolysis and therefore increase independence from mycoplasma or viruses, which can cause a series of effects in mitochondrial energy supplies. Maintenance of a cell model the infected cells. For a long time, the most reliable test to in either low or high glucose conditions is therefore likely to identify mycoplasma contamination involved their cultivation influence experimental treatments that interfere with energy and detection by microbiological assays, requiring a testing metabolism. Another often ignored factor is the gender of the period of about one month (Rottem and Barile, 1993). These cell donor (De Souza Santos et al., 2018). receptors prolonged tests have been largely replaced by a PCR-based are found on most cell types and exposure of male cells to technique that has emerged as the method of choice for fast estrogenic media (e.g. phenol red structurally resembles and reliable detection of mycoplasma (Nübling et al., 2015). some nonsteroidal ) can lead to estrogen-receptor- To avoid mycoplasma infections, the best way to handle newly dependent influences on cell proliferation, differentiation, arrived cells is to thaw and cultivate them in quarantine until and metabolism (Berthois et al., 1986; Farzaneh and Zarghi, the testing results arrive. It is also important to strictly avoid 2016). In female cells, culture conditions can lead to a release of storage of non-tested cells in liquid nitrogen tanks where the epigenetic inactivation of one of the two X- there is potential for direct physical contact between the cells and, consequently, the expression of gene products from both and the nitrogen. Furthermore, routine testing of applied cell X-chromosomes, coding for a large number of genes involved lines should be conducted several times per year to ensure

Frontiers in Pharmacology | www.frontiersin.org 5 December 2019 | Volume 10 | Article 1484 Hirsch and Schildknecht In Vitro Reproducibility Crisis

early detection of contamination that might occur in everyday descriptions. Normalization usually starts with definition of routine work. The introduction of a fast and reliable test for control values as 100% and all other data are consequently mycoplasma detection has significantly lowered the burden indicated as percentages of the control values. The inclusion of contamination inspections. Hence, mycoplasma testing and definition of 0% values is however often ignored, but

should be a mandatory element in any cell culture laboratory. is equally relevant, e.g. for the calculation of IC50 values. In 7. Reporting: An obvious reason for poor reproducibility cell-based toxicological investigations, sigmoidal curves are is insufficient, incomplete, and inaccurate reporting of usually obtained by a four-parameter fit (lower and upper methodologies. Although detailed guidelines on GCCP and asymptotes, turning point, and steepness of the curve at the reporting standards have been published, a brief glimpse turning point). Automatic fitting often leads to curves that at the materials and methods section of submitted and fail to hit the 100% bar due to problems with control values. published manuscripts clearly reveals that there is still a long In such cases, re-normalization procedures are required, and difficult road ahead. Far too often, the materials and e.g. to determine benchmark responses, and 2–3 points methods section becomes the first casualty when journal in the no-effect range could alternatively be applied for word count limits urge authors to scale down manuscripts. re-normalization (Krebs et al., 2018). Such interventions, On the other hand, it is commonly accepted that the quality however, can only be justified on the basis of a researcher’s of reporting methodologies ranks among the most important profound knowledge of the biology of the experimental elements for enhancing reproducibility. In plain words, good model and the test assay applied. Krebs and colleagues reporting covers all the information another researcher needs recently published an annotated template for the description to exactly reproduce an . Initiatives fostering a of cell-based toxicological test methods, based on the OECD standardization of documentation required for cell models guidance document 211 (GD211) (Krebs et al., 2019). The culminated in the definition of GCCP standards (Coecke et al., template provides a clearly defined structure intended to 2005; Hartung et al., 2019), which were recently modified to support researchers in the design of their study and in meet requirements that have emerged with the advent of stem handling of the data obtained. It has been explicitly compiled cell research (Pamies et al., 2017). GCCP guidance focuses in user-oriented fashion to gain greater acceptance by the on the following principles: 1) understanding the in vitro research community. Finally, the practice of "reporting model; 2) quality of materials and methods; 3) documentation the best results" only and ignoring those experiments that of information necessary to allow repetition of the work; 4) failed to yield the expected outcome has an impact on protection of the environment and individuals from hazards; data reproducibility. Some journals already request the 5) compliance with laws, regulations, and ethical principles; publication of all raw data, e.g. displaying entire western and 6) training of staff to ensure quality work and safety. blots instead of cut-out bands of interest. This aspiration The primary focus of the GCCP principles is documentation could be amended by the publication of all experiments of the test models, and initiatives to standardize method performed to address a defined question, even those documentation have resulted in the introduction of SOPs. that are not included in the main body of a manuscript. Currently, there is no obligatory organization of SOPs in Whether this attempt to include all data would actually the field ofin vitro toxicology. Initiatives such as the EURL be implemented by researchers and publishers is a whole ECVAM DataBase service on Alternative Methods to Animal other question. Experimentation (https://ecvam-dbalm.jrc.ec.europa.eu) have compiled a large and steadily growing collection of protocols organized in a standardized and uniform manner. Depending DISCUSSION AND OUTLOOK on the acceptance and distribution of these protocols within the scientific community, their organization could serve as a solid The necessity for enhancement of result reproducibility in the life platform for the generation of new SOPs relevant to in vitro sciences has been identified and is gradually being internalized research in the future. The formulation and application of SOPs by researchers and institutions alike (Prinz et al., 2011; Drucker, is undoubtedly a central element to improving experimental 2016). Achieving high rates of reproducibility often stands reproducibility. Thoughtless use of SOPs, however, could lead in contradiction to the discovery of novel scientific insights. to insufficient integration of new knowledge into existing Although it is obvious that new findings are only of use if they methods and must be guarded against. Reproducibility is a can be reproduced, Dirnagel recently cautioned that cutting- central aspect of life science research, but it must not prevent edge discovery is unavoidably associated with a high rate of false progress. Centralized collections of SOPs and methods need positive results (Dirnagl, 2019). These false positive results are to guarantee precise documentation of high quality methods an integral part of exploration and must not be conflated with but should also allow modifications of protocols through a intentional manipulation of results. structured process of discussion and consensus among experts Even with the best of intentions, it must be concluded that in respective fields to ensure both innovation and replication. the limits of reproducibility in cell culture work is reached when Appropriate and complete reporting, however, does not end confronted with the question of reference standards, particularly at the raw data level. Data handling further influences result for established and widely distributed cell lines. Simply put, which reproducibility. For example, the use of either absolute or of the currently available and characterized stocks of common normalized data is often not adequately indicated in study cell lines, like HeLa cells, should be considered as the gold

Frontiers in Pharmacology | www.frontiersin.org 6 December 2019 | Volume 10 | Article 1484 Hirsch and Schildknecht In Vitro Reproducibility Crisis standard? Even if a consensus could be reached for individual not gained the attention they deserve from researchers and cell lines, storage capacity limitations force even large cell banks publishers alike. The question therefore arises how consensus to passage their cells, which necessarily influences the cells in one on a standard for a given cell line can be achieved and way or another over time. how its application can be motivated. This task can only be We have written the present article as a condensed accomplished by members of the communities regularly using introduction of effective, easy-to-implement measures to improve a given cell line. The formulation of new guideline protocols reproducibility of experimental results. In plain summary, the should be fostered by the respective scientific societies. Well most relevant rules are: established and influential laboratories might play a key role to reach consensus on a ready-to-use standard protocol for - Ensure an in-depth understanding of the cell model by the the handling of a given cell line. Application of these protocols researchers working with it; a thorough characterization is should in a next step become mandatory for the acceptance a prerequisite to identify the validity of a cell model and of a new study by the scientific community, unless deviations its limits. from the standard protocol can be scientifically justified. Such - Pay attention to seemingly irrelevant details; the selection measures will not bring overwhelming scientific merit for the of plasticware, coating, or the type of medium, can have individual scientist, but are inevitable steps to re-establish and significant influence on the outcome of a study as summarized maintain confidence of both researchers and the general public in Figure 1. in contemporary biomedical in vitro research. - Be aware of what an experimental readout actually detects. Viability, for instance, is routinely assessed by different assays (e.g. resazurin reduction, LDH release, ATP detection) but AUTHOR CONTRIBUTIONS they measure different cellular parameters and hence cannot be compared directly. CH and SS contributed equally to the preparation of the - Provide all the information in your reporting that is necessary manuscript. to precisely reproduce your experiment.

Above all the aspects discussed, one of the most influential FUNDING factors in any attempt to improve reproducibility is a researcher’s consolidated knowledge about the cell model in We acknowledge funding from the NanoScreen Materials use (see Principle 1 of GCCP; Coecke et al., 2005). The in-depth Challenge co-funded by the Competence Centre for Materials characterization of cellular parameters relevant to a given Science and Technology (CCMX) as well as support by scientific question is certainly a resource-consuming endeavor, the Doerenkamp-Zbinden foundation, the Land-BW (N but is a worthwhile investment in the long run, both in the EURODEG), the BMBF (NeuriTox) and by the Projects from selection of an appropriate cell model and interpretation of the European Union’s Horizon 2020 research and innovation the results. As the residence time of scientists in laboratories program EU-ToxRisk (grant agreement No 681002). is often limited, the quality and continuity of their supervision by experienced staff becomes a critical factor in knowledge transfer. The second of the most influential factors contributing ACKNOWLEDGMENTS to insufficient reproducibility is the lack of generally accepted and mandatory guidelines on the cultivation of individual cell We would like to thank Editage (www.editage.com) for the lines. Guidance documents such as the GCCP or GIVIMP, English language editing. We thank Prof. Dr. Marcel Leist and have been published for quite some time, but so far have Dr. Peter Wick for the critical reading of the manuscript.

REFERENCES Baharvand, H., Hashemi, S. M., Kazemi Ashtiani, S., and Farrokhi, A. (2006). Differentiation of human embryonic stem cells into hepatocytes in 2D and Abdal Dayem, A., Lee, S., Choi, H., and Cho, S. G. (2018). The impact of adhesion 3D culture systems in vitro. Int. J. Dev. Biol. 50 (7), 645–652. doi: 10.1387/ molecules on the In Vitro culture and differentiation of stem cells. Biotechnol. J. ijdb.052072hb 13 (2), 1700575. doi: 10.1002/biot.201700575 Baker, M. (2016). 1,500 scientists lift the lid on reproducibility.Nature 533 (7604), Adeleye, Y., Andersen, M., Clewell, R., Davies, M., Dent, M., Edwards, S., et al. 452–454. doi: 10.1038/533452a (2015). Implementing Toxicity Testing in the 21st Century (TT21C): making Battiston, K. G., McBane, J. E., Labow, R. S., and Paul Santerre, J. (2012). Differences safety decisions using toxicity pathways, and progress in a prototype risk in binding and cytokine release from monocytes on commercially assessment. Toxicology 332, 102–111. doi: 10.1016/j.tox.2014.02.007 sourced tissue culture polystyrene. Acta Biomater. 8 (1), 89–98. doi: 10.1016/j. Al-Ani, A., Toms, D., Kondro, D., Thundathil, J., Yu, Y., and Ungrin, M. (2018). actbio.2011.09.015 Oxygenation in cell culture: critical parameters for reproducibility are routinely Begley, C. G., and Ellis, L. M. (2012). Drug development: raise standards for not reported. PloS One 13 (10), e0204269. doi: 10.1371/journal.pone.0204269 preclinical cancer research. Nature 483 (7391), 531–533. doi: 10.1038/483531a Ankley, G. T., Bennett, R. S., Erickson, R. J., Hoff, D. J., Hornung, M. W., Johnson, Berthois, Y., Katzenellenbogen, J. A., and Katzenellenbogen, B. S. (1986). Phenol R. D., et al. (2010). Adverse outcome pathways: a conceptual framework to red in tissue culture media is a weak estrogen: implications concerning the support ecotoxicology research and risk assessment. Environ. Toxicol. Chem. study of estrogen-responsive cells in culture. Proc. Natl. Acad. Sci. U.S.A. 83 29 (3), 730–741. doi: 10.1002/etc.34 (8), 2496–2500. doi: 10.1073/pnas.83.82496

Frontiers in Pharmacology | www.frontiersin.org 7 December 2019 | Volume 10 | Article 1484 Hirsch and Schildknecht In Vitro Reproducibility Crisis

Bohmer, N., Rippl, A., May, S., Walter, A., Heo, M. B., Kwak, M., et al. (2018). García-Parra, P., Maroto, M., Cavaliere, F., Naldaiz-Gastesi, N., Álava, J. I., García, Interference of engineered nanomaterials in flow cytometry: a . A. G., et al. (2013). A neural extracellular matrix-based method for in vitro Colloids Surf B. Biointerfaces 172, 635–645. doi: 10.1016/j.colsurfb.2018.09.021 hippocampal neuron culture and dopaminergic differentiation of neural stem Brodbeck, W. G., Nakayama, Y., Matsuda, T., Colton, E., Ziats, N. P., and Anderson, cells. BMC Neurosci. 14, 48. doi: 10.1186/1471-2202-14-48 J. M. (2002). Biomaterial surface chemistry dictates adherent monocyte/ Giancotti, F. G., and Ruoslahti, E. (1999). Integrin signaling. Science 285 (5430), macrophage cytokine expression in vitro. Cytokine 18 (6), 311–319. doi: 1028–1032. doi: 10.1126/science.285.54301028 10.1006/cyto.20021048 Guggenheim, E., Milani, S., Roettgermann, P., Dusinska, M., Saout, C., Salvati, A., Brook, I. (1989). Inoculum effect. Rev. Infect. Dis. 11 (3), 361–368. doi: 10.1093/ et al. (2018). Refining in vitro models for nanomaterial exposure to cells and clinids/11.3.361 tissues. NanoImpact 10, 121–142. doi: 10.1016/j.impact.2018.02.008 Capes-Davis, A., Theodosopoulos, G., Atkin, I., Drexler, H. G., Kohara, A., Gutbier, S., May, P., Berthelot, S., Krishna, A., Trefzer, T., Behbehani, M., et al. MacLeod, R. A., et al. (2010). Check your cultures! A list of cross-contaminated (2018). Major changes of cell function and toxicant sensitivity in cultured cells or misidentified cell lines.Int. J. Cancer 127 (1), 1–8. doi: 10.1002/ijc.25242 undergoing mild, quasi-natural genetic drift.Arch. Toxicol. 92 (12), 3487–3503. Clinchy, B., Reza Youssefi, M., and Håkansson, L. (2003). Differences in adsorption doi: 10.1007/s00204-018-2326-5 of serum proteins and production of IL-1ra by human monocytes incubated in Hartung, T., and Sabbioni, E. (2011). Alternative in vitro assays in nanomaterial different tissue culture microtiter plates.J. Immunol. Methods 282 (1-2), 53–61. toxicology. Wiley Interdiscip. Rev. Nanomed Nanobiotechnol. 3 (6), 545–573. doi: 10.1016/j.jim.2003.07.013 doi: 10.1002/wnan.153 Coecke, S., Balls, M., Bowe, G., Davis, J., Gstraunthaler, G., Hartung, T., et al. Hartung, T., De Vries, R., Hoffmann, S., Hogberg, H. T., Smirnova, L., Tsaioun, K., (2005). Guidance on good cell culture practice. a report of the second ECVAM et al. (2019). Toward good in vitro reporting standards. ALTEX 36 (1), 3–17. task force on good cell culture practice. Altern. Lab. Anim. 33 (3), 261–287. doi: doi: 10.14573/altex.1812191 10.1177/026119290503300313 Hirsch, C., Roesslein, M., Krug, H. F., and Wick, P. (2011). Nanomaterial cell Collins, F. S., and Tabak, L. A. (2014). Policy: NIH plans to enhance reproducibility. interactions: are current in vitro tests reliable? Nanomedicine (Lond.) 6 (5), Nature 505 (7485), 612–613. doi: 10.1038/505612a 837–847. doi: 10.2217/nnm.11.88 Collins, F. S., Gray, G. M., and Bucher, J. R. (2008). Toxicology. Transforming Hughes, P., Marshall, D., Reid, Y., Parkes, H., and Gelber, C. (2007). The costs of environmental health protection. Science 319 (5865), 906–907. doi: 10.1126/ using unauthenticated, over-passaged cell lines: how much more data do we science.1154619 need? Biotechniques 43 (5), 575, 577–8. doi: 10.2144/000112598 Damsky, C. H., and Ilić, D. (2002). Integrin signaling: it’s where the action is. Curr. Karlsson, H., Fryknäs, M., Larsson, R., and Nygren, P. (2012). Loss of cancer drug Opin. Cell Biol. 14 (5), 594–602. doi: 10.1016/S0955-0674(02)00368-X activity in colon cancer HCT-116 cells during spheroid formation in a new 3-D De Souza Santos, R., Frank, A. P., Palmer, B. F., and Clegg, D. J. (2018). Sex and spheroid cell culture system. Exp. Cell Res. 318 (13), 1577–1585. doi: 10.1016/j. media: considerations for cell culture studies. ALTEX 35 (4), 435–440. doi: yexcr.2012.03.026 10.14573/altex.1806151 Kilkenny, C., Parsons, N., Kadyszewski, E., Festing, M. F., Cuthill, I. C., Fry, D., Didion, J. P., Buus, R. J., Naghashfar, Z., Threadgill, D. W., Morse, H. C., and de et al. (2009). Survey of the quality of experimental design, statistical analysis Villena, F. P. (2014). SNP array profiling of mouse cell lines identifies their and reporting of research using animals. PloS One 4 (11), e7824. doi: 10.1371/ strains of origin and reveals cross-contamination and widespread aneuploidy. journal.pone.0007824 BMC 15, 847. doi: 10.1186/1471-2164-15-847 Kleensang, A., Vantangoli, M. M., Odwin-DaCosta, S., Andersen, M. E., Dike, L. E., and Farmer, S. R. (1988). Cell adhesion induces expression of growth- Boekelheide, K., Bouhifd, M., et al. (2016). Genetic variability in a frozen batch associated genes in suspension-arrested fibroblasts.Proc. Natl. Acad. Sci. U. S. of MCF-7 cells invisible in routine authentication affecting cell function. Sci. A. 85 (18), 6792–6796. doi: 10.1073/pnas.85.186792 Rep. 6, 28994. doi: 10.1038/srep28994 Dirnagl, U. (2019). Rethinking research reproducibility. EMBO J. 38 (2), e101117. Krebs, A., Nyffelerm, J., Rahnenführer, J., and Leist, M. (2018). Normalization of doi: 10.15252/embj.2018101117 data for viability and relative cell function curves. ALTEX 35 (2), 268–271. doi: Drucker, D. J. (2016). Never waste a good crisis: confronting reproducibility 10.14573/1803231 in translational research. Cell Metab. 24 (3), 348–360. doi: 10.1016/j. Krebs, A., Waldmann, T., Wilks, M., van Vugt-Lussenburg, B., van der Burg, B., cmet.2016.08.006 Terron, A., et al. (2019). Template for the description of cell-based toxicological Edmondson, R., Broglie, J. J., Adcock, A. F., and Yang, L. (2014). Three- test methods to allow evaluation and regulatory use of the data". ALTEX 36 (4), dimensional cell culture systems and their applications in and 682–699. doi: 10.14573/altex.1909271 cell-based . Assay Drug Dev. Technol. 12 (4), 207–218. doi: 10.1089/ Laurent, S., Burtea, C., Thirifays, C., Häfeli, U. O., and Mahmoudi, M. (2012). adt.2014.573 Crucial ignored parameters on nanotoxicology: the importance of toxicity Elliott, J. T., Rösslein, M., Song, N. W., Toman, B., Kinsner-Ovaskainen, A., assay modifications and "cell vision".PloS One 7 (1), e29997. doi: 10.1371/ Maniratanachote, R., et al. (2017). Toward achieving harmonization in a nano- journal.pone.0029997 cytotoxicity assay measurement through an interlaboratory comparison study. Leist, M., Ghallab, A., Graepel, R., Marchan, R., Hassan, R., Bennekou, S., et al. ALTEX 34 (2), 201–218. doi: 10.14573/altex.1605021 (2017). Adverse outcome pathways: opportunities, limitations and open Errington, T. M., Iorns, E., Gunn, W., Tan, F. E., Lomax, J., and Nosek, B. A. (2014). questions. Arch. Toxicol. 91 (11), 3477–3505. doi: 10.1007/s00204-017-2045-3 An open investigation of the reproducibility of cancer biology research. eLife 3, Libby, P. (2015). Murine "model" monotheism: an iconoclast at the altar of mouse. e04333. doi: 10.7554/eLife.04333 Circ. Res. 117 (11), 921–925. doi: 10.1161/CIRCRESAHA.115.307523 Eskes, C., Boström, A. C., Bowe, G., Coecke, S., Hartung, T., Hendriks, G., et al. Liberio, M. S., Sadowski, M. C., Soekmadji, C., Davis, R. A., and Nelson, C. C. (2017). Good cell culture practices and in vitro toxicology. Toxicol. In Vitro 45 (2014). Differential effects of tissue culture coating substrates on prostate (Pt 3), 272–277. doi: 10.1016/j.tiv.2017.04.022 cancer cell adherence, morphology and behavior. PloS One 9 (11), e112122. Farzaneh, S., and Zarghi, A. (2016). Estrogen receptor ligands: a review (2013- doi: 10.1371/journal.pone.0112122 2015). Sci. Pharm. 84 (3), 409–427. doi: 10.3390/scipharm84030409 Longhurst, C. M., and Jennings, L. K. (1998). Integrin-mediated signal Fiegel, H. C., Havers, J., Kneser, U., Smith, M. K., Moeller, T., Kluth, D., et al. (2004). transduction. Cell Mol. Life Sci. 54 (6), 514–526. doi: 10.1007/s000180050180 Influence of flow conditions and matrix coatings on growth and differentiation McKee, T. J., and Komarova, S. V. (2017). Is it time to reinvent basic cell culture of three-dimensionally cultured rat hepatocytes. Tissue Eng. 10 (1-2), 165–174. medium? Am. J. Physiol. Cell Physiol. 312 (5), C624–C626. doi: 10.1152/ doi: 10.1089/107632704322791817 ajpcell.003362016 Frattini, A., Fabbri, M., Valli, R., De Paoli, E., Montalbano, G., Gribaldo, L., et al. Nübling, C. M., Baylis, S. A., Hanschmann, K. M., Montag-Lessing, T., Chudy, M., (2015). High variability of genomic instability and gene expression profiling in Kreß, J., et al. (2015). World Health Organization International Standard different HeLa clones.Sci. Rep. 5, 15377. doi: 10.1038/srep15377 To Harmonize Assays for Detection of Mycoplasma DNA. Appl. Environ. Freedman, L. P., Cockburn, I. M., and Simcoe, T. S. (2015). The economics of Microbiol. 81 (17), 5694–5702. doi: 10.1128/AEM.01150-15 reproducibility in preclinical research. PloS Biol. 13 (6), e1002165. doi: 10.1371/ Natl. Res Counc. (NRC). (2007). Toxicity testing in the 21st Century: A Vision and journal.pbio.1002165 a Strategy (Washington, DC: Natl. Acad. Press).

Frontiers in Pharmacology | www.frontiersin.org 8 December 2019 | Volume 10 | Article 1484 Hirsch and Schildknecht In Vitro Reproducibility Crisis

Niepel, M., Hafner, M., Mills, C. E., Subramanian, K., Williams, E. H., Chung, M., Schrurs, F., and Lison, D. (2012). Focusing the research efforts.Nat. Nanotechnol. et al. (2019). A multi-center study on the reproducibility of drug-response assays 7 (9), 546–548. doi: 10.1038/nnano.2012.148 in mammalian cell lines. Cell Syst. 9 (1), 35–48.e5. doi: 10.1016/j.cels.2019.06.005 Shah, K., McCormack, C. E., and Bradbury, N. A. (2013). Do you know the Ohnuma, T., Arkin, H., and Holland, J. F. (1986). Effects of cell density on drug- sex of your cells? Am. J. Physiol. Cell Physiol. 306 (1), C3–18. doi: 10.1152/ induced cell kill kinetics in vitro (inoculum effect).Br. J. Cancer 54 (3), 415– ajpcell.002812013 421. doi: 10.1038/bjc.1986.192 Shen, M., and Horbett, T. A. (2001). The effects of surface chemistry and Olson, H., Betton, G., Robinson, D., Thomas, K., Monro, A., Kolaja, G., et al. adsorbed proteins on monocyte/macrophage adhesion to chemically (2000). Concordance of the toxicity of pharmaceuticals in humans and in modified polystyrene surfaces.J. BioMed. Mater Res. 57 (3), 336–345. doi: animals. Regul. Toxicol. Pharmacol. 32 (1), 56–67. doi: 10.1006/rtph.20001399 10.1002/1097-4636(20011205)57:3<336::AID-JBM1176>3.0.CO;2-E Orlowska, A., Perera, P. T., Kobaisi, M., Dias, A., Nguyen, H. K. D., Ghanaati, S., Somaiah, C., Kumar, A., Mawrie, D., Sharma, A., Patil, S. D., Bhattacharyya, J., et al. (2017). The effect of coatings and nerve growth factor on attachment and et al. (2015). Collagen promotes higher adhesion, survival and proliferation differentiation of pheochromocytoma cells.Materials (Basel) 11 (1), E60. doi: of Mesenchymal Stem Cells. PloS One 10 (12), e0145068. doi: 10.1371/journal. 10.3390/ma11010060 pone.0145068 Pamies, D., Bal-Price, A., Simeonov, A., Tagle, D., Allen, D., Gerhold, D., et al. Sun, M., Chi, G., Li, P., Lv, S., Xu, J., Xu, Z., et al. (2018). Effects of matrix stiffness (2017). Good cell culture practice for stem cells and stem-cell-derived models. on the morphology, adhesion, proliferation and osteogenic differentiation of ALTEX 34 (1), 95–132. doi: 10.1016/j.toxlet.2017.07.196 mesenchymal stem cells. Int. J. Med. Sci. 15 (3), 257–268. doi: 10.7150/ijms.21620 Pamies, D., Bal-Price, A., Chesné, C., Coecke, S., Dinnyes, A., Eskes, C., et al. Takemura, Y., Kobayashi, H., Miyachi, H., Hayashi, K., Sekiguchi, S., and (2018). Advanced good cell culture practice for human primary, stem cell- Ohnuma, T. (1991). The influence of tumor cell density on cellular accumulation derived and organoid models as well as microphysiological systems. ALTEX 35 of doxorubicin or cisplatin in vitro. Cancer Chemother. Pharmacol. 27 (6), 417– (3), 353–378. doi: 10.14573/altex.1710081 422. doi: 10.1007/BF00685154 Petersen, E. J., Hirsch, C., Elliott, J. T., Krug, H. F., Aengenheister, L., Arif, A. T., Teppola, H., Sarkanen, J. R., Jalonen, T. O., Linne, M. L. (2018). Impacts of laminin et al. (2019). Cause-and-effect analysis as a tool to improve the reproducibility and polyethyleneimine surface coatings on morphology of differentiating of nanobioassays: four case studies. Chem. Res. Toxicol. doi: 10.1021/acs. human SH-SY5Y cells and networks. In: Eskola H., Väisänen J., Viik J., chemrestox.9b00165 Hyttinen J., editors. EMBEC and NBC 2017. EMBEC 2017, NBC 2017. IFMBE Pöltl, D., Schildknecht, S., Karreman, C., and Leist, M. (2012). Uncoupling of ATP- Proceedings, 65. Singapore: Springer. doi: 10.1007/978-981-10-5122-7_75 depletion and cell death in human dopaminergic neurons. Neurotoxicology 33 Terron, A., Bal-Price, A., Paini, A., Monnet-Tschudi, F., Bennekou, S. H.EFSA WG (4), 769–779. doi: 10.1016/j.neuro.2011.12.007 EPI1 Members, et al. (2018). An adverse outcome pathway for parkinsonian Prats-Mateu, B., Ertl, P., and Toca-Herrera, J. L. (2014). Influence of HepG2 cell motor deficits associated with mitochondrial complex I inhibition.Arch. shape on nanoparticle uptake. Microsc. Res. Tech. 77 (8), 560–565. doi: 10.1002/ Toxicol. 92 (1), 41–82. doi: 10.1007/s00204-017-2133-4 jemt.22374 U.S. Environ. Prot. Agency (EPA). (2009). The U.S. Environmental Protection Prinz, F., Schlange, T., and Asadullah, K. (2011). Believe it or not: how much can Agency’s strategic plan for evaluating the toxicity of chemicals. http://www. we rely on published data on potential drug targets? Nat. Rev. Drug Discovery epa.gov/spc/toxicitytesting/. 10 (9), 712. doi: 10.1038/nrd3439-c1 Vinci, M., Gowan, S., Boxall, F., Patterson, L., Zimmermann, M., Court, W., et al. Rösslein, M., Elliott, J. T., Salit, M., Petersen, E. J., Hirsch, C., Krug, H. F., et al. (2015). (2012). Advances in establishment and analysis of three dimensional tumor Use of cause-and-effect analysis to design a high-quality nanocytotoxicology spheroid-based functional assays for target validation and drug evaluation. assay. Chem. Res. Toxicol. 28 (1), 21–30. doi: 10.1021/tx500327y BMC Biol. 10, 29. doi: 10.1186/1741-7007-10-29 Rottem, S., and Barile, M. F. (1993). Beware of . Trends Biotechnol. 11 Voelkl, B., Vogt, L., Sena, E. S., and Würbel, H. (2018). Reproducibility of (4), 143–151. doi: 10.1016/0167-7799(93)90089-R preclinical animal research improves with heterogeneity of study samples. PloS Saravia, V., and Toca-Herrera, J. L. (2009). Substrate influence on cell shape and Biol. 16 (2), e2003693. doi: 10.1371/journal.pbio.2003693 cell mechanics: HepG2 cells spread on positively charged surfaces. Microsc. Res. Wörle-Knirsch, J. M., Pulskamp, K., and Krug, H. F. (2006). Oops they did it again! Tech. 72 (12), 957–964. doi: 10.1002/jemt.20742 Carbon nanotubes hoax scientists in viability assays. Nano Lett. 6 (6), 1261– Schildknecht, S., Pöltl, D., Nagel, D. M., Matt, F., Scholz, D., Lotharius, J., et al. 1268. doi: 10.1021/nl060177c (2009). Requirement of a dopaminergic neuronal phenotype for toxicity of low Wright Muelas, M., Ortega, F., Breitling, R., Bendtsen, C., and Westerhoff, H. V. concentrations of 1-methyl-4-phenylpyridinium to human cells. Toxicol. Appl. (2018). Rational cell culture optimization enhances experimental reproducibility Pharmacol. 241 (1), 23–35. doi: 10.1016/j.taap.2009.07.027 in cancer cells. Sci. Rep. 8 (1), 3029. doi: 10.1038/s41598-018-21050-4 Schildknecht, S., Pape, R., Müller, N., Robotta, M., Marquardt, A., Bürkle, A., et al. (2011). Neuroprotection by minocycline caused by direct and specific Conflict of Interest: The authors declare that the research was conducted in the scavenging of peroxynitrite. J. Biol. Chem. 286 (7), 4991–5002. doi: 10.1074/jbc. absence of any commercial or financial relationships that could be construed as a M110.169565 potential conflict of interest. Schildknecht, S., Pape, R., Meiser, J., Karreman, C., Strittmatter, T., Odermatt, M., et al. (2015). Preferential extracellular generation of the active parkinsonian Copyright © 2019 Hirsch and Schildknecht. This is an open-access article toxin mpp+ by transporter-independent export of the intermediate MPDP+. distributed under the terms of the Creative Commons Attribution License Antioxid. Redox Signal. 23 (13), 1001–1016. doi: 10.1089/ars.20156297 (CC BY). The use, distribution or reproduction in other forums is permitted, Scholz, D., Pöltl, D., Genewsky, A., Weng, M., Waldmann, T., Schildknecht, S., provided the original author(s) and the copyright owner(s) are credited and et al. (2011). Rapid, complete and large-scale generation of post-mitotic that the original publication in this journal is cited, in accordance with accepted neurons from the human LUHMES cell line. J. Neurochem. 119 (5), 957–971. academic practice. No use, distribution or reproduction is permitted which does doi: 10.1111/j.1471-4159.2011.07255.x not comply with these terms.

Frontiers in Pharmacology | www.frontiersin.org 9 December 2019 | Volume 10 | Article 1484