The Power of Replicates

Technical Note: Gene Expression Profiling The Power of Replicates Introduction Figure 1: Differentially Expressed Genes As A Function Of Replicates Carefully designing and controlling experiments is as important as the execution of the experiment itself. One approach that ensures greater experimental success in gene expression studies using microarrays is 7000 the incorporation of replicates. Replication of conditions lends statistical power that increases the confidence of the conclusions drawn 6000 from these experiments. This text discusses the many ways in which s researchers can benefit from using replicates in their studies. 5000 ssed Gene In a typical gene expression study, researchers are interested in genes re 4000 expressed above background levels, and genes that are differentially expressed between conditions of interest. The variation present in mi- 3000 entially Exp croarray data poses the challenge of determining whether differences er ff between expression measurements are caused by biological differ- Di 2000 ences, or by statistical chance. The best way to address this challenge is to use replicates for each condition studied. There are two primary 1000 types of replicates: technical and biological. Technical replicates 12 3 4 56 involve taking one sample from the same source tube, and analyzing it Number of samples in Each Group across multiple conditions, e.g., analyzing one sample six times across multiple arrays. Biological replicates are different samples measured Increasing the number of biological replicates in each group enhances the across multiple conditions, e.g., six different human samples across power to detect differentially expressed genes. six arrays. Using replicates offers three major advantages: significant differential expression. If the data distribution is unclear, • Replicates can be used to measure variation in the experiment non-parametric tests such as the Mann-Whitney test can be applied. so that statistical tests can be applied to evaluate differences. Several publications make specific recommendations on the number • Averaging across replicates increases the precision of gene of replicates required to detect various fold changes3,4. expression measurements and allows smaller changes to be detected. B. Increase Precision • Replicates can be compared to locate outlier results Averaging across replicates enhances the precision of measurements. that may occur due to aberrations within the array, If the standard deviation of an expression measurement is s, then the sample, or the experimental procedure. the standard deviation of the average across n replicates is. As the number of replicates increases, both the detectable difference from background and the detectable fold change decrease. The high cost of microarrays has typically constrained or eliminated the number of replicates in most studies. However, the cost must be C. Detect Outliers evaluated against the quality of the data, which includes ease of use, The presence of outlier samples can have a severe impact on the initial financial outlay, cost of arrays and reagents, and experimental interpretation of data. Most array platforms have internal controls that design (e.g., replicates assayed). A more informative experiment may detect various problems in an experiment. However, internal con- be achieved by assaying a smaller set of test conditions while includ- trols may not identify all issues. A more powerful approach is also to ing more replicates rather than assaying a larger set of test conditions consider the correlation between replicates. Subtle problems with the with fewer replicates. array, the sample, or the experimental procedure often become obvi- ous in a pair-wise plot of replicate measurements. BENEFITS A. Measure Variation Data Replicates improve the measurement of variation. Normally, if only one To illustrate the points above, a data set of 12 samples were analyzed array exists per condition, then fold change is used to determine dif- on the Illumina Human Whole-Genome Expression BeadChips. The ferential expression. However, the variation of the expression level for samples included six biological replicates from normal tissue and each gene is different and unknown. Multiple studies have shown that six biological replicates from diseased tissue. Figure 1 illustrates the fold change on its own is an unreliable indicator1,2. If multiple measure- number of differentially expressed genes as a function of the number ments (i.e., replicates) exist for each gene within each condition, the of samples in each group. When the number of samples was two or measurement of variation can be estimated. If the data follow more, a standard t-test was applied with a false positive rate of 0.05. an approximately normal distribution, the t-test or its variants reveal For one sample in each group, fold change was used to determine dif- Technical Note: Gene Expression Profiling Figure 2: Identification Of Outliers 6 2 4 r = 0.97 ) 4 0 2 mal Sample 2 ( x1 Nor 0 -2 -1 012 3 45 Normal Sample 1 ( x104 ) 6 4 r 2 = 0.97 ) 4 0 2 Outlier Sample 1 ( x1 0 -2 -1 012 3 45 Normal Sample 1 ( x104 ) By plotting correlation plots between replicates, outlier samples may be identified (top) that are not apparent without a replicate comparison (bottom). Technical Note: Gene Expression Profiling Table 1: Differently Expressed Genes Identified Using the t-test, we can estimate the noise in the data, so that less Under Various Conditions differential expression was detected between conditions, but the false positive rate remained the same. Figure 2 shows that correlation plots Between Between Normal between replicates would have revealed the outlier sample. Normal Samples And Disease Fold change, 554 1,906 Conclusion no outlier With the increased density of current microarrays, gene expression t-test: 3 replicates, no 559 3,377 researchers must be able to discern between true differences across outlier experimental and controlled samples, and those caused by random Fold change 1,898 3,259 variation. As the above data demonstrate, an experimental design with outlier that employs replicates allows researchers to detect variation within t-test: 3 replicates with 503 2,640 the assay, increases the precision of measurement when comparing outlier data points, and provides the ability to detect misleading and irrelevant outliers. ferential expression. The fold change required for significance was set References to achieve the same false positive rate as that used in the t-test. 1. Chen Y, Dougherty ER, Bittner ML (1997) Ratio-based decisions and the quantitative analysis of cDNA microarray images. J Biomed Optics 2: 364- The effect and treatment of outliers on a data set can be illustrated as 367. follows. Normal samples were divided into two groups of three. Three 2. Newton MA, Kendziorski CM, Richmond CS, Blattner FR, Tsui KW (2001) disease samples were selected. Random noise was added to one On differential variability of expression ratios: improving statistical inference of the normal samples to simulate an outlier, and differential expres- about gene expression changes from microarray data. Journal of Compara- sion was assessed between the two groups of normal samples and tive Biol 8: 37-52. between the normal and disease samples. Differential expression was 3. Pan W, Lin J, Le CT (2002) How many replicates of arrays are required to determined via fold change and then by t-test employing replicates. detect gene expression changes in microarray experiments? A mixture Table 1 shows the number of differentially expressed genes identified model approach. Genome Biol 3 (5) Epublication. under the various conditions. 4. Tibshirani R, A simple method for assessing sample sizes in microarray experiments. www-stat.stanford.edu/~tibs/ftp/samplesize.pdf. Genes differentially expressed between normal samples were false positives. In the fold-change analysis with no replicates, the false positive rate increases dramatically if the sample analyzed was an outlier. Illumina, Inc. • 9885 Towne Centre Drive, San Diego, CA 92121 USA • 1.800.809.4566 toll-free • 1.858.202.4566 tel • [email protected] • illumina.com FOR RESEARCH USE ONLY © 2010 Illumina, Inc. All rights reserved. Illumina, illuminaDx, Solexa, Making Sense Out of Life, Oligator, Sentrix, GoldenGate, GoldenGate Indexing, DASL, BeadArray, Array of Arrays, Infinium, BeadXpress, VeraCode, IntelliHyb, iSelect, CSPro, GenomeStudio, Genetic Energy, HiSeq, and HiScan are registered trademarks or trademarks of Illumina, Inc. All other brands and names contained herein are the property of their respective owners. Pub. No. 470-2006-006 Current as of 13 Sep 2007.

Load more