The Choice of Housekeeping Gene and Relative Efficiency of Paired T-Tests in RT-PCR Data Analysis

The Choice of Housekeeping Gene and Relative Efficiency of Paired t-tests in RT-PCR Data Analysis

Thesis

Presented in Partial Fulfillment of the Requirements for the Degree Master of Science in the Graduate School of The Ohio State University

Yi Guo

Graduate Program in Public Health

The Ohio State University

2010

Thesis Committee:

Michael Pennell, Advisor

Soledad Fernandez

Yi Guo

2010 Abstract

Real time polymerase chain reaction (RT-PCR), a method of quantitatively measuring gene expression, is widely used in biomedical research. It is a highly sensitive technique which can accurately measure the presence of even trace amounts of mRNA in the cell.

To produce reliable results, it is essential to use stably expressed housekeeping genes for data normalization. In this study, a RT-PCR dataset that contains 12 cancer related genes from 16 different tumor tissues and 3 housekeeping genes was analyzed. Black raspberry was used as a treatment against cancer and its effects on the expression of these 12 target genes were examined using a paired design. The objective of this study was to examine the effect of normalization on efficiency of the t-test and the choice of housekeeping gene

(HKG) on the identification of differentially expressed genes in RT-PCR experiments.

Data analysis showed that results differed with choice of HKG since 2 of the 3 HKGs were differentially expressed across treatment groups. We also found that normalization improved the efficiency of the t-test for most of the target genes.

Acknowledgements

I would like to thank my advisor, Dr. Michael Pennell, for his guidance during my master studies. Dr. Pennell has always been encouraging me to think independently through his method of teaching, which has truly helped me in my scientific career. Dr. Pennell also spent a lot of time on the revision of this thesis. Without his guidance, this thesis would not be as nice as it is now.

I would also like to thank Dr. Soledad Fernandez for serving on my thesis committee.

She spent a lot of time on my thesis and came up with some new ideas for future research in this area.

iii

Vita

November 29, 1980...... Born – Wuhan, Hubei, China

June 2002...... Bachelor of Science in Biochemistry, Wuhan University.

2004 – 2009...... Doctor of Philosophy in Biochemistry, The Ohio State University.

Publications

Guo, Y. , Yuan, C., Tian, F., Huang, K., Weghorst, C. M., Tsai, M.-D., and Li, J. (2010) Contributions of Conserved TPLH Tetrapeptides to the Conformational Stability of Ankyrin Repeat Proteins. J Mol Biol 399(1), 168-81.

Guo, Y. , Yuan, C., Weghorst, C. and Li, J. (2010) IKKβ specifically binds to P16 and phosphorylates Ser8 of P16. Biochem Biophys Res Commun 393(3), 504-508.

Li, J. and Guo, Y . (2010) Gankyrin Oncoprotein: Structure, Function, and Involvement in Cancer. Current Chemical Biology 4(1), 13-19.

Guo, Y. , Mahajan, A., Yuan, C., Joo, S. H., Weghorst, C. M., Tsai, M.-D., and Li, J. (2009) Comparisons of the Conformational Stability of Cyclin-Dependent Kinase (CDK) 4-Interacting Ankyrin Repeat (AR) Proteins. Biochemistry 48, 4050-4062.

Mahajan, A.†, Guo, Y. †, Yuan, C., Weghorst, C. M., Tsai, M.-D., and Li, J. (2007) Dissection of protein-protein interaction and CDK4 inhibition in the oncogenic versus tumor suppressing functions of gankyrin and p16. J Mol Biol 373, 990-1005.

Fields of Study

Major Field: Public Health iv

Table of Contents

Abstract...... ii

Acknowledgement...... iii

Vita...... iv

List of Tables...... vii

List of Figures...... viii

Chapter 1: Introduction...... 1

Chapter 2: Introduction to RT-PCR and Data Analysis...... 3

2.1 Real Time Polymerase Chain Reaction (RT-PCR)...... 3

2.2 Quantification of mRNA levels using the delta-delta Ct method...... 5

2.3 Normalization in RT-PCR data analysis...... 7

Chapter 3: Data Analysis and the Choice of Housekeeping Gene...... 10

3.1 Introduction...... 10

3.2 Methods...... 11

3.3 Results...... 12

3.4 Discussion...... 15

Chapter 4: The Effect of Normalization and Pairing on Efficiency in RT-PCR Studies. 18

4.1 Introduction...... 18 v

4.2 Methods...... 20

4.3 Results...... 22

4.4 Discussion...... 27

Chapter 5: Conclusion...... 28

References...... 31

List of Tables

Table 1. Fold changes in target gene expression and p-values of paired t-tests...... 13

Table 2. Fold changes in HKG expression and p-values of paired t-tests...... 14

Table 3. The relative efficiencies for paired design versus unpaired design...... 23

Table 4. Relative efficiencies of t-tests using normalized Ct values relative to raw Ct values in a paired or an unpaired design...... 24

Table 5. A demonstration of property 4.1 using HKG1...... 26

vii

List of Figures

Figure 1. Exponential amplification of DNA in PCR...... 4

viii

CHAPTER 1

Introduction

Real time polymerase chain reaction (RT-PCR) is a highly sensitive technique of

monitoring the expression of genes in real time. Since it allows accurate measurement of

lowly expressed genes, RT-PCR has become a standard method in almost all cell biology

labs. Despite its high sensitivity, RT-PCR has many problems. For example, recent

studies have shown that inappropriate normalization of RT-PCR data using invalidated

housekeeping genes (HKGs) could lead to biased results (1-5).

In this study, we investigated two issues related to the use of HKGs in RT-PCR data analysis. First, we examined the problem of choosing the right housekeeping gene in RT-

PCR data analysis using data from a chemoprevention study. Black raspberry was used to inhibit cell replication and modify gene expression in human oral tumor cells (6). The investigator was interested in studying how the berry treatment would affect the expression of 12 oral cancer related genes. Three different widely used HKGs were included in the RT-PCR experiments by the investigator. We compared the methods of normalization by normalizing the data in two different ways: 1. Normalization using each of the three HKGs. 2. Normalization using the average expression of these three HKGs.

After normalization, differential expression was analyzed using paired t-test on all 12

1 target genes and 3 HKGs.

Next, we examined the effect of normalization and pairing on analysis of

RT-PCR studies by calculating and comparing the relative efficiency of the t-test under different experimental designs and normalization methods. Relative efficiency was calculated by comparing the standard errors of the difference in Ct values before treatment and after treatment.

The reminder of this paper is organized as follows: in chapter 2, we provide an introduction to RT-PCR and its data analysis. Previous publications on RT-PCR data normalization are also reviewed. In chapter 3, we examine the effects of choice of

HKG on data analysis. Recommendations are made on how to choose the proper HKG.

In chapter 4, efficiencies of the t-test under different scenarios (normalized or non-normalized, paired or unpaired) are calculated and compared. In chapter 5, we draw conclusions of our findings and make recommendations for analyzing RT-PCR data in future research.

2 CHAPTER 2

Introduction to RT-PCR and Data Analysis

2.1 Real Time Polymerase Chain Reaction (RT-PCR).

Real time polymerase chain reaction (RT-PCR), also called quantitative real time PCR (Q-PCR), is a molecular biology technique that allows one to amplify and quantify a DNA sequence in one single step. In a PCR experiment, two primers

(oligonucleotides) that are complementary to the two ends of a predetermined sequence on each of the two strands of the target DNA are used. These primers can bind to the target DNA sequence, usually one gene, with high specificity. After the binding of primers to the target DNA (template), a thermo-stable DNA polymerase also binds to the template and extends the primers by adding deoxyribonucleotides

(building blocks for all DNA) to the end of them. After one round of amplification, the same primers are used again in the next round, in which they can bind not only to the original template DNA strands but also to the DNA made in the previous round of synthesis. Thus, exponential amplification of the desired DNA sequence in the template DNA is achieved (Figure 1 ).

Figure 1 . Exponential amplification of DNA in PCR (7).

Using RT-PCR, scientists can monitor the expression of target genes in real time.

Gene expression is regulated by controlling the amount of gene transcripts, messenger

RNA (mRNA), in the cells of all organisms. In a RT-PCR experiment, mRNA is extracted from the cells and reverse transcribed into complementary DNA (cDNA) by the enzyme reverse transcriptase. Then the cDNA of the target genes is amplified in a

PCR reaction. This allows accurate detection of gene expression from very small amounts of mRNA present in the cells. Fluorescence dyes or probes are also added to the samples to identify gene expression. Fluorescence dyes fluoresce only when bound to the double stranded DNA (i.e., the PCR product), and hence the concentration of the PCR product can be determined by measuring the level of fluorescence after each cycle. Fluorescent reporter probes are short DNA sequences complimentary to the target DNA sequences (i.e., the target genes). They have a

4 fluorescence reporter on one end and a fluorescence quencher on the other end. Under normal conditions, the reporter does not fluoresce due to the presence of the quencher

(reporter quenched). However, during the PCR process, the 5'-3'-exonuclease activity of the DNA polymerase degrades the probe and separates the reporter from the quencher, resulting in an increase in fluorescence which is recorded and used for calculating the concentrations of the PCR product. Typical dyes or probes used in

RT-PCR experiments include the TagMan® probe, the SYBR green DNA dye, and the molecular beacons.

2.2 Quantification of mRNA levels using the delta-delta Ct method.

There are several methods for measuring the alterations in mRNA levels using

RT-PCR, such as the standard curve method (8), the Pfaffl analysis methods (9) and the delta-delta Ct method (10). In this study, we focus on the most commonly used delta-delta Ct method.

In each reaction of RT-PCR, usually in a well on a 96-well plate, the expression intensity (fluorescence from the probe) of a certain gene is measured. This measurement is called Cycles to Threshold (Ct), which is defined as the number of cycles needed for the amount of amplified DNA to reach the threshold level set by the researcher. A larger Ct value means more PCR cycles are needed for the amount of

DNA to reach the threshold, which indicates that there is a smaller amount of mRNA in the sample. The delta-delta Ct method involves comparing the Ct values of the

5 samples of interest with a calibrator (control) such as a sample from normal tissue.

The Ct values of both the target samples and the calibrator are normalized to an appropriate endogenous housekeeping gene.

In RT-PCR, the amount of DNA after n cycles (X n ) is:

= + n X n X 0 1( E) (1)

where X 0 is the amount of DNA initially and E is the efficiency of the amplification.

Let CtX , CtC , and CtH denote the number of threshold cycles for the target sample,

control sample and HKG respectively and X ctX , C ctC and H ctH denote the corresponding amounts of DNA. From equation (1), we have:

= + CtX X CtX X 0 1( Ex ) (2)

= + CtC CCtC C0 1( EC ) (3)

= + CtH H CtH H 0 1( EH ) (4)

where X 0 , C 0 and H 0 are the initial amount of DNA in the target sample, the

calibrator, and the housekeeping gene respectively and E x , E c , and E H are the efficiencies for the amplification reaction for each. In the delta-delta Ct method, the

efficiency of the amplification is assumed to be 1 for all reactions (i.e., E x = E c =

n E H = 1). Thus, the increase will be 2 after n cycles and equations 2-4 can be re-written as:

= × CtX X CtX X 0 2 (5)

= × CtC CCtC C0 2 (6)

6 = × CtH H CtH H 0 2 (7)

To normalize the DNA expression in the target sample to that in the housekeeping gene, equation (5) is divided by equation (7):

CtX X X × 2 X − CtX = 0 = 0 × 2CtX CtH (8) × CtH H CtH H 0 2 H 0

X Also, since CtX = K is a constant, we have: H CtH

X 0 × 2CtX −CtH = K (9) H 0

X ∆− = 0 = × Ct 1 X N K 2 (10) H 0 where ∆Ct 1 = CtX − CtH . Therefore, the relative amount of target to a control group, both normalized to the endogenous housekeeping gene is given by:

∆− Ct 1 X K × 2 ∆− ∆− ∆∆− N = = 2 ( Ct 1 Ct )2 = 2 Ct (11) × ∆− Ct 2 C N K 2

where X N and C N are the normalized amounts of DNA in the target sample and the calibrator sample respectively.

2.3 Normalization in RT-PCR data analysis

In RT-PCR data analysis, gene expression data are normalized to eliminate sample-to-sample variation. Materials (cells or tissues) obtained from different individuals are normally quite different in cell numbers or administration of treatment, so the amount of mRNA present is also different. Moreover, different reverse

7 transcription and PCR efficiencies could further increase variability (11). Thus, it is necessary to use controls called housekeeping genes (HKGs) in RT-PCR experiments for standardization. Theoretically, the usage of HKGs ensures that all the steps in a

RT-PCR experiment are controlled for (12).

The ideal HKGs are genes that are expressed at relatively constant levels regardless of experimental conditions. Usually, HKGs are genes that are required for the maintenance of basal cellular function. For example, GAPDH and β-action are widely used HKGs.

Accurate normalization of RT-PCR data is essential for getting reliable results, especially when the differences in gene expression are very small. Unfortunately, it remains one of most difficult problems in RT-PCR since it is very difficult to pick the right housekeeping gene to use.

Many investigators use well-known housekeeping genes in their RT-PCR experiments. However, these genes were historically used in semi-quantitative techniques such as Northern blots. When a highly sensitive quantitative technique like

RT-PCR is used, the appropriateness of these housekeeping genes must be re-evaluated since the expression of these genes cannot be considered constant any more. As early as 1975, it has been shown that the expression of 18sRNA could be increased by cytomegalovirus infection (13). It has also been shown that the expression of GAPDH was different in different rat tissues (14). As more experiments are being done on these famous housekeeping genes, it has become increasingly clear

8 that the expression of these genes does not remain unchanged under all conditions.

Sometimes the changes in expression are huge, which makes the most-popular housekeeping genes unsuitable for normalization (15-17). Bas et al showed that for gene expression analysis of human T lymphocytes at different activation stages, only

18sRNA was a reliable HKG, whereas GAPDH mRNA was increased by 30-70 folds upon activation of these cells (15). Bémeur et al showed that under hyperglycemic conditions in a rat model of focal cerebral ischemia, β-actin mRNA expression was decreased compared with that under normal conditions (16). Similarly, Schmid et al showed that GAPDH was not consistently expressed in microdissected human renal biopsies from 165 different patients (17). Thus far, no HKGs have been proven to have invariable expression under all experimental conditions (1-5). Therefore, several different HKGs should be included in a RT-PCR experiment to avoid false results caused by improper normalization.

9 CHAPTER 3

Data Analysis and the Choice of Housekeeping Gene

3.1 Introduction

The analysis of gene expression using RT-PCR has become essential in biomedical research. However, even though RT-PCR is a highly sensitive technique, it suffers from certain drawbacks. For instance, inappropriate data normalization is one of the main problems in RT-PCR data analysis.

In RT-PCR experiments, normalization is usually done by co-amplifying housekeeping genes (HKGs) with the target genes. Since the HKGs go through the same experimental procedures as the target genes, it is thought that every step of the

PCR experiment is controlled for. Thus, an ideal housekeeping gene should be consistently expressed under all experimental conditions. HKGs affected by the experimental conditions are not reliable and they can lead to biased results.

Currently, many of the widely used HKGs are traditionally used in semi-quantitative methods like Northern blot. When a highly sensitive method like

RT-PCR is used, we should re-evaluate the usage of these genes since many of them were shown to be unreliable under certain experimental conditions. In this chapter, we

10 examine how different normalization methods may affect the results of RT-PCR using data from a cancer chemoprevention study. We draw conclusions from these results and discuss how investigators should choose HKGs in light of our findings.

3.2 Methods

Data

It has been found that antioxidants like anthocyanins in fruits and vegetables may be protective against many different types of cancers (18-20). For example, it was shown that anthocyanins in black raspberries could prevent esophageal tumors in rats (20). In our motivating experiment, lyophilized black raspberry was used as a treatment against oral cancer as it is believed that it may affect the expression of genes involved in the formation of oral tumors. The goal of the study was to identify genes that are targets of the treatment by comparing the mRNA expression of 12 target genes in tumor tissues before treatment and after treatment for 16 patients in a paired design. Three different housekeeping genes (HKGs) were included in the RT-PCR experiments for the purpose of normalization.

Analysis

Paired t-tests were used to determine if the expression of the target genes was affected by the berry treatment. The overall significance level was set at α = 0.05. For each gene of interest, the hypotheses are:

11 µ = µ H 0 : T 2 T1

µ ≠ µ H a : T 2 T1

µ where T 2 is the mean normalized Ct values for the sample after the treatment and

µ T1 is the mean normalized Ct values for the sample before the treatment. Three different HKGs were used for normalization. We considered normalization using each gene separately as well as the average Ct across genes. In this study, we assume normality is satisfied, and all patients are independent of each other. Holm’s method was used to control for multiple testing (21).

3.3 Results

The fold changes (treated vs. untreated) and the p-values from each of the paired t-tests are summarized in Table 1 .

12 Gene HKG1 HKG2 HKG3 HKGs averaged Fold p-value Fold p-value Fold p-value Fold p-value Change Change Change Change Gene1 0.36 0.2920 1.75 0.8262 2.49 0.0803 1.16 1.0000 Gene2 1.44 1.0000 7.11 0.5280 10.08 0.1053 4.69 0.6369 Gene3 0.50 0.3474 2.46 0.8262 3.48 0.1256 1.62 1.0000 Gene4 0.44 0.0253 2.18 0.1968 3.09 0.0768 1.44 0.4944 Gene5 0.13 0.3388 0.64 1.0000 0.91 0.8954 0.42 1.0000 Gene6 0.61 1.0000 2.99 0.7460 4.24 0.1020 1.97 1.0000 Gene7 0.39 0.0253 1.91 0.8262 2.71 0.2288 1.26 1.0000 Gene8 0.28 0.0096 1.27 1.0000 1.93 0.2110 0.88 1.0000 Gene9 0.26 0.0253 1.20 1.0000 1.83 0.4656 0.83 1.0000 Gene10 0.43 0.6316 1.99 0.8262 3.03 0.1778 1.38 1.0000 Gene11 0.88 1.0000 4.04 0.8262 6.18 0.1878 2.80 1.0000 Gene12 0.40 0.4055 1.96 0.8262 2.78 0.2288 1.29 1.0000 Table 1 . Fold changes in target gene expression and p-values of paired t-tests.

Based on the adjusted p-values of the paired t-tests in Table 1 , after normalization using HKG1, the mRNA expression of gene 4 (p-value = 0.0253), gene

7 (p-value = 0.0023), gene 8 (p-value = 0.0096), and gene 9 (p-value = 0.0253) was affected by the berry treatment. The expression of mRNA for these genes after treatment was only 0.44, 0.39, 0.28, and 0.26 of that before treatment respectively.

However, based on the paired t-test, the mRNA expression of all 12 genes were unaffected by treatment if HKG2, HKG3 or the average Ct of all three HKGs was used for normalization.

It is also noticeable that the changes in the expression of the same gene can even be in opposite directions when different HKGs are used. For example, when HKG1 is used, the fold change in gene 4 expression is 0.44 which means that it is

13 down-regulated by the treatment. However, when HKG2 is used, the fold change in gene 4 expression is 2.18 which means it is up-regulated by the treatment. The results following normalization using HKG2 and HKG3 contradict the results following normalization using HKG1 for all 12 genes except for gene 2 and gene 5. If the average Ct of all three HKGs is used for normalization, the results contradict the results from HKG1 for all genes except for gene 2, 5, 8, and 9 ( Table 1 ).

In light of these results, we examined whether the expression of the HKGs was affected by the berry treatment. For each of the three HKGs, a paired t-test was used to test the following hypotheses:

µ= µ H0: N 2 N 1

µ≠ µ H a: N2 N 1

µ µ where N 2 is the mean Ct value after the treatment and N1 is the mean Ct value before the treatment. The p-values for the three t-tests are summarized in Table 2 .

µ −µ Genes Fold Changes ( 2 T 2 T 1 ) p-value HKG1 0.92 0.9303 HKG2 0.19 0.1589 HKG3 0.13 0.0580

Table 2 . Fold changes in HKG expression and p-values of paired t-tests.

The tests of HKGs 2 and 3 are marginally significant (p-values = 0.1589 and

14 0.058 respectively), while the test of HKG1 is highly insignificant (p-value 0.9303).

Thus, HKG1 is least affected by the berry treatment. Furthermore, the fold change in the expression levels of HKG1 is 0.92, which is very close to 1, whereas the fold changes for HKGs 2 and 3 are far away from 1 (0.19 and 0.13 respectively).

3.4 Discussion

In our data analysis, among the three well-known HKGs, only HKG1 was stable across the experimental conditions. The expression level of HKG2 and HKG3 after treatment was only 0.19 and 0.13 of that before treatment. Moreover, normalization using the average effect of all HKGs did not give significant results. This is due to the fact that all HKGs were affected by the treatment in the same direction (all were decreased by the treatment) and hence averaging does not balance out the differential expression in these HKGs. Therefore, HKG1 should be used for normalization for this dataset since its expression level is least affected by the treatment among all three

HKGs. For this dataset, we can conclude that among all twelve genes related to oral cancer, the expression of genes 4, 7, 8 and 9 are affected by the berry treatment.

Further studies on using black raspberry as a treatment against oral cancer could focus on these four genes.

Nowadays, different investigators use different methods to choose housekeeping genes for their RT-PCR experiments. The most widely used methods are the following three: the use of well-known traditional housekeeping genes like GAPDH, β-actin and

15 18s rRNA, the use of the average expression level of several well-known housekeeping genes, and the use of genes that are least affected by the experimental conditions.

Previous studies have shown that not all well-known HKGs are consistently expressed under all experimental conditions as they are thought to be. Our results have once again confirmed this by showing that only one of the three HKGs used was reliable. Glyceraldehyde 3-phosphate dehydrogenase (GAPDH), one of the most-popular HKGs, would not be a suitable HKG in this experiment since its expression could be altered by the black raspberry treatment. Black raspberries contain large amounts of glucose whereas GAPDH is an enzyme that breaks down glucose to generate energy. An increase in glucose in the cell would probably boost the expression of GAPDH. Thus, it is essential to validate HKGs, such as GAPDH, before using them for normalization.

Some investigators include a panel of multiple well-known housekeeping genes

(usually 3-5) in their RT-PCR experiments and use the average expression level of these genes for normalization. It is hoped that the differential expression of the housekeeping genes would be balanced out by this averaging. However, this method is still problematic because it takes time and money to find the perfect combination of housekeeping genes if there is one. Moreover, if an investigator uses many different experimental conditions in an RT-PCR experiment, there is no guarantee that the expression of a panel of housekeeping genes remains unchanged under all these

16 experimental conditions. Also note that if treatment affects all or most HKGs in one direction (as in our study), averaging will not balance out the differential expression.

Therefore, to ensure normalization is valid and the results are unbiased, it is a good idea to validate the housekeeping genes in all RT-PCR experiments. The expression of several housekeeping genes should be compared across conditions and the gene least affected by treatment should be used for normalization. Investigators should not simply use well-known HKGs without validation.

17 CHAPTER 4

The Effect of Normalization and Pairing on Efficiency in RT-PCR Studies

4.1 Introduction

In any experiment, there are many sources of experimental errors, such as experimental procedures, measurement procedures, and experimental designs. To obtain consistent and meaningfully results, errors from these sources must be controlled in all studies. In RT-PCR, high quality experimental procedures are essential for reducing experimental errors. The success of a RT-PCR experiment depends on how good a researcher can carry out the experimental procedures like sample preparation and sample transferring. In a poorly performed experiment, the variance of the Ct values could be huge and the estimated treatment means could be biased. A good experimental design could also reduce experimental errors in RT-PCR; a blocked or paired design could greatly reduce variability due to differences in genetics and physiology between animal or human subjects.

Paired designs are commonly used to control experimental error. In a paired design, each subject is measured under two different treatment conditions. On the contrary, subjects in different treatment groups are assumed to be independent in an

18 unpaired design. In animal or human studies, between subject differences can be very large, making it difficult to detect between treatment differences. Paired designs control for between subject variability by examining the effects of treatments on each subject, thereby making it easier to detect a treatment effect when it exists.

One can also show mathematically why pairing often reduces variability in an

experiment. Let X 1 and X 2 be the mean response in treatment groups 1 and 2 respectively. Assuming constant variance across treatment groups (i.e.,

σ2= σ 2 = σ 2 1 2 ), the variance of the difference in means is given by: 2 VarX(−=+− X ) VarX () VarX ()2(,) CovX X =−σ2 (1) ρ (12) 12paired 1 2 12 n where ρ is the correlation between the paired measurements on an individual subject.

For an unpaired design, we assume the two groups are independent ( ρ = 0): 2 VarX(− X ) = VarX ()() + VarX = σ 2 (13) 12unpaired 1 2 n Therefore, the relative efficiency of the paired design versus the unpaired design is:

2 σ 2 Var( X− X ) 1 RE =1 2 unpaired =n = (14) Var( X− X )2 2 1 − ρ 1 2 paired σ(1− ρ ) n

In a paired design, paired measurements are usually positively correlated ( ρ > 0) and hence RE > 1; i.e., the paired design reduces the variability of the estimated difference in means compared to the unpaired design.

In a RT-PCR experiment, normalization using properly validated HKGs can also improve efficiency. By running HKGs side by side with your target genes in the experiment, every step in RT-PCR (e.g., sample preparation and sample loading) is

19 considered controlled for, thereby reducing variability due to experimental error.

In the last chapter, we found that the results of RT-PCR are sensitive to the choice of HKGs. Since pairing already minimizes the errors due to biological variability in an experiment, one would question whether normalization is still necessary if a paired design is used. In this chapter, we studied whether normalization can further improve efficiency in a paired design. The same RT-PCR dataset as in chapter 3 was used. Each subject was measured twice, before the berry treatment and after the berry treatment in a paired design. Efficiencies were calculated for the paired design and the unpaired design (i.e. ignoring the pairing of the correct design) before normalization and after normalization. Our study investigates whether HKGs could be omitted from a paired design by studying whether normalization could further reduce experimental errors in such a paired design.

4.2 Methods

Data

As described in chapter 3, the dataset we considered contains information on expression of 12 different genes from 16 different patients before berry treatment and after berry treatment in a paired design. Note here that in this study, all samples from the same subject were run on sample plate so pairing both reduces biological and experimental variability (caused by plate to plate differences). Three different housekeeping genes were used for normalization.

20 Analysis

In order to evaluate the effects of pairing and normalization on efficiency, we compared standard errors of the difference between treated and untreated means under four different scenarios:

i. Unpaired t-test, without normalization

ii. Paired t-test, without normalization

iii. Unpaired t-test, with normalization

iv. Paired t-test, with normalization

To compare the efficiency of these four conditions, we calculated the relative efficiencies (RE). The relative efficiency of condition 1 (C1) relative to condition 2

(C2) is defined as:

Se (C ) Relative Efficiency (C1, C2) = 2 (15) Se (C1 )

Where Se( C 1 ) and Se( C 2 ) are the standard errors of the t-test under two different

scenarios (e.g., C1 = paired t using raw Ct values; C2 = unpaired t using raw Ct

values). If, in this case, RE ( C1 ,C2 ) was greater than 1, it would mean that pairing decreased variability thereby increasing the power to detect a difference across treatment conditions.

A paired design was used in the original experiment. In order to compare the RE of the paired design relative to the unpaired design, we ignored the natural pairing in

21 the dataset and computed the standard error of the difference for an unpaired (equal variance) t-test. We calculated standard errors for both the paired and unpaired tests before and after normalization with each of the 3 HKGs. Relative efficiencies were used to answer the following questions:

1) Does pairing improve efficiency (i.e. is RE (ii, i) >1)?

2) Does normalization improve efficiency in an unpaired design (i.e. is RE (iii, i) > 1)?

3): Does normalization improve efficiency in a paired design (i.e. is RE (iv, ii) >

1)?

4.3 Results

First, the effect of pairing on the relative efficiency was examined. RE (ii, i), paired design versus unpaired design, was calculated to determine whether pairing could improve efficiency. The relative efficiencies and the Pearson’s correlations between un-normalized Ct values before treatment (T1) and after treatment (T2) are provided in Table 3 .

22 Gene ID RE (ii, i) Correlation (T2, T1) 1 0.9421 – 0.1325 2 0.9155 – 0.2081 3 0.9770 – 0.0557 4 0.9945 – 0.0132 5 0.9901 – 0.0214 6 1.1791 0.2838 7 0.8343 – 0.4841 8 0.9371 – 0.1509 9 1.1842 0.3521 10 0.9257 – 0.1784 11 1.0280 0.0702 12 0.9466 – 0.1167

Table 3 . The relative efficiencies for paired design versus unpaired design.

As seen in Table 3 , the values of RE (i, ii) for all 12 genes are very close to 1 except for a few of genes (genes 6, 7, and 9), which indicates that the t-tests in a paired design have about the same efficiency as the t-tests in an unpaired design.

Interestingly, all genes except for genes 6, 9 and 11 have relative efficiencies smaller than 1. This means that the paired design is actually less efficient than the unpaired design. By checking the Pearson’s correlations between the Ct values before treatment and after treatment, we found the correlations corresponding to these genes are negative which explains why the paired design has worse efficiency (Table 3 column

3).

Table 4 contains relative efficiencies comparing normalized and raw data for both the paired and unpaired t-tests. Relative efficiencies under the three different

23 housekeeping genes are listed separately. From the table we can see that in both the paired and unpaired designs, the relative efficiencies are greater than 1 for all genes except gene 2 and gene 11. For about half of the genes, the relative efficiencies are greater than 2 and some of the REs are even greater than 4. For example, for gene 4, the relative efficiency of the normalized Ct relative to raw Ct ranges from 2.5226 to

4.0074 in both paired and unpaired designs depending on the HKG used. This indicates that normalization is improving the efficiencies of the t-test for most of the genes in both designs.

Paired t tests , normalized data Unpaired t tests , normalized relative to raw data data relative to raw data Gene ID HKG1 HKG2 HKG3 HKG1 HKG2 HKG3 1 1.9320 3.0517 2.6856 2.2545 3.2250 3.4610 2 0.4029 0.3313 0.3500 0.5051 0.3521 0.3983 3 1.5866 1.2147 1.4199 1.9961 1.2003 1.4788 4 2.8479 3.3465 2.8323 4.0074 3.1230 2.5226 5 1.4949 1.7309 1.9353 1.3407 1.4695 1.8101 6 1.3442 1.3588 1.4351 1.6187 1.4580 1.6952 7 1.8038 1.6904 1.5553 3.7743 2.4678 1.9966 8 2.4356 2.4694 2.8568 2.9426 2.3399 3.0808 9 2.2966 1.3885 1.6058 1.8610 1.1834 1.4083 10 1.9710 2.0356 1.9024 1.9394 2.5975 2.4498 11 0.8882 0.6364 0.6756 0.8825 0.5621 0.6099 12 1.4309 1.6303 1.5964 2.2335 2.2242 2.2217 Table 4. Relative efficiencies of t-tests using normalized Ct values relative to raw Ct values in a paired or an unpaired design.

24 For gene 2 and gene 11, the relative efficiencies are smaller than 1, which shows that the variances of the differences between the estimated means before and after treatment are actually increased after normalization. This unusual finding is due to the following property:

Property 4.1 : Normalization increases the standard error of a paired or unpaired t-test if:

1 Var (H ) ρ < j (16) 2 Var (X i )

where X i is the raw Ct values for gene of interest i and H j is the raw Ct values for

housekeeping gene j, and Var (X i ) and Var (H j ) are the variances of the raw Ct values for the gene of interest and the housekeeping gene, respectively.

To prove this property, the following formulas were derived:

− = + − Var (X i H j ) Var (X i ) Var (H j ) 2Cov (X i , H j ) (17)

− where Var (X i H j ) is the variance of the normalized Ct values for each individual

gene, and Cov (X i , H j ) is the covariance between X i and H j .

For gene 2 and gene 11, since the relative efficiencies are smaller than 1, we have:

− > Var (X i H j ) Var (X i ) (18)

− − = − > Var (X i H j ) Var (X i ) Var (H j ) 2Cov (X i , H j ) 0 (19)

It can be shown that:

= ρ Cov (X i , H j ) Var (X i )Var (H j ) (20)

ρ where is the correlation between X i and H j . Combining formula (19) and (20), we

25 have:

− ρ > Var (H j ) 2 Var (X i )Var (H j ) 0 (21)

1 Var (H ) ρ < j 2 Var (X i )

1 Var (H ) In Table 5, the variances of the raw differences, ρ, and j are 2 Var (X i ) provided for HKG1. Not surprisingly, formula (16) is satisfied only by genes 2 and 11

due to the small variances of the raw Ct values of these genes; Var( H 2 ) and

Var( H 1 ) are < ½ variance of all other genes.

Gene Variance of raw Correlation ( ρ) 1 Var (H ) ID difference j 2 Var (X ) i 1 34.3930 0.4399 0.8964 2 3.4256 1.3940 0.8704 3 15.2891 0.6598 0.9436 4 26.9401 0.4971 0.9687 5 53.8906 0.3515 0.6670 6 23.0939 0.5369 0.8248 7 31.8822 0.4569 0.9656 8 27.5216 0.4918 0.9414 9 15.5365 0.6546 0.9262 10 39.7769 0.4091 0.8577 11 7.2794 0.9563 0.8820

12 40.4749 0.4055 0.8984

Table 5. A demonstration of property 4.1 using HKG1.

26 4.4 Discussion

By examining the relative efficiencies of the t-tests, it was shown that pairing may not always improve efficiency. When the correlation between the raw Ct values before treatment and after treatment is negative, a paired design is less efficient than an unpaired design. It was also shown that normalization can improve efficiency in both the paired design and unpaired design unless the variance of the mean difference in raw Ct values is already very small. For the most part, normalization using all three housekeeping genes decreased the standard error which increased the efficiency of the t-tests in our sample data. However, the results for gene 2 and gene 11 demonstrated that when the variability of the raw Ct values is small, normalization can actually increase the standard error of the difference in group means and hence reduce efficiency.

Overall, normalization using HKGs could further reduce variability and increase efficiency of t-test in the presence of pairing. Biological variability could be controlled by pairing, however, variability in experimental procedures like sample preparation and sample loading could only be controlled by normalization. Thus, it is essential to always include HKGs in RT-PCR experiments for data analysis.

27 CHAPTER 5

Conclusion

In chapter 3, we investigated how the choice of housekeeping gene (HKG) could affect the results in RT-PCR data analysis using a real RT-PCR dataset. Data normalization was done using each of the three HKGs and the average expression of the three. It was shown that when HKG1 was used, the expression of 4 out of the 12 target genes was significantly affected by the treatment. However, when the other two

HKGs and the average expression were used for normalization, none of target genes were differentially expressed. Since the three popular HKGs gave inconsistent results, we also checked whether the expression of these HKGs was altered by the treatment.

Based on the data analysis, only HKG1 was stable across experimental conditions, while the expression of the other two HKGs was dramatically changed. Therefore, under the conditions used in this experiment, HKG1 is the only reliable HKG and should be used for normalization.

In chapter 4, we examined how normalization could affect relative efficiency of a paired design. By calculating the relative efficiencies of the t-tests, we have provided evidence that pairing does not always improve efficiency because the paired

28 measurements may be negatively correlated. We have also provided evidence that normalization improves efficiency unless the variance of the gene of interest is already small.

In summary, we have proven that normalization using HKGs could generally improve efficiency of the t-test in a paired or unpaired design in RT-PCR data analysis.

However, choosing the right HKG to use could be problematic since many well-known HKGs were shown to be unreliable which could lead to completely opposite results ( Table 1 ). Based on our results and previous findings, we make the following two recommendations for investigators who use RT-PCR in their studies: first, one should validate HKGs for each different experimental condition they use by including several HKGs in the experiment and examining if their expression changes.

Only HKGs stably expressed across conditions should be used for data normalization.

Secondly, one should always include HKGs in their RT-PCR experiments and perform normalization for data analysis. Experimental errors are inevitable even in well-designed experiments; normalization could further minimize these experimental errors.

As for future work, one could develop statistical methodology for analyzing

RT-PCR data which address some of the issues presented in this paper. For example, models could be built to allow for potential changes in HKGs across treatment groups.

By developing such models, one could potentially perform normalization without having to worry about differential expression of HKGs producing biased treatment

29 effects.

Another question one could explore is whether technical replicates (e.g., replication of measurements and/or experiments) could minimize errors in experimental procedures so that normalization is no longer required. For example, to obtain technical replicates for the black raspberry study, tissue samples from the same subject would be preserved and run under the same treatment condition (i.e., both before and after berry treatment) in identical RT-PCR experiments on different days.

The result from each experiment would be considered a technical replicate. Ideally, experimental errors can be balanced out by repeating the same observation several times and using the average measurements of the replicates. Using a RT-PCR dataset with technical replicates and HKGs, we could calculate relative efficiencies of the t-test under different scenarios to examine whether using technical replicates could improve the efficiency of the t-test. If normalization using HKGs does not improve efficiency in the presence of technical replicates, then there would be no need for investigators to take the extra effort selecting the perfect HKGs for their RT-PCR experiments.

30 References

1. Dheda K, Huggett JF, Bustin SA, Johnson MA, Rook G, Zumla A. Validation of housekeeping genes for normalizing RNA expression in real-time PCR. Biotechniques 2004; 37: 112–119.

2. Tricarico C, Pinzani P, Bianchi S et al. Quantitative real-time reverse transcription polymerase chain reaction: normalization to rRNA or single housekeeping genes is inappropriate for human tissue biopsies. Anal Biochem 2002; 309: 293–300.

3. Ullmannova V, Haskovec C. The use of housekeeping genes (HKG) as an internal control for the detection of gene expression by quantitative real-time RT-PCR. Folia Biol (Praha) 2003; 49: 211–216.

4. Koch I, Weil R, Wolbold R et al. Interindividual variability and tissue-specificity in the expression of cytochrome P450 3A mRNA. Drug Metab Dispos 2002; 30: 1108–1114.

5. Bär M, Bär D, Lehmann B. election and validation of candidate housekeeping genes for studies of human keratinocytes--review and recommendations. J Invest Dermatol . 2009 Mar;129(3):535-7.

6. Stoner GD, Wang LS, Casto BC. Laboratory and clinical studies of cancer chemoprevention by antioxidants in berries. Carcinogenesis 2008 Sep;29(9):1665-74.

7. Andy Vierstraete 1999 (http://users.ugent.be/~avierstr/principles/pcr.html).

8. Livak, K. 1997. ABI Prism 7700 Sequence Detection System, User Bulletin 2. PE Applied Biosystems, Foster City, CA.

9. Pfaffl, M.W. 2001. A new mathematical model for relative quantification in real-time RT-PCR. Nucleic Acids Res . 29:e45.

10. Livak, K.J. and T.D. Schmittgen. 2001. Analysis of relative gene expression data using real-time quantitative PCR and the 2(-Delta Delta C(T)) method. Methods 25:402-408.

31 11. Bustin SA, Nolan T. Pitfalls of quantitative real-time reversetranscription polymerase chain reaction. J Biomol Tech 2004; 15: 155–166.

12. Huggett J, Dheda K, Bustin S, Zumla A. Real-time RT-PCR normalisation; strategies and considerations. Genes Immun . 2005 Jun; 6(4):279-84.

13. Tanaka S, Furukawa T, Plotkin SA. Human cytomegalovirus stimulates host cell RNA synthesis. J Virol 1975; 15: 297–304.

14. Piechaczyk M, Blanchard JM, Marty L et al. Post-transcriptional regulation of glyceraldehyde-3-phosphate-dehydrogenase gene expression in rat tissues . Nucleic Acids Res 1984; 12: 6951–6963.

15. Bas A, Forsberg G, Hammarstrom S, Hammarstrom ML. Utility of the housekeeping genes 18S rRNA, beta-actin and glyceraldehyde-3-phosphate-dehydrogenase for normalization in real-time quantitative reverse transcriptase-polymerase chain reaction analysis of gene expression in human T lymphocytes. Scand J Immunol 2004; 59: 566–573.

16. Bémeur C, Ste-Marie L, Desjardins P et al. Decreased beta-actin mRNA expression in hyperglycemic focal cerebral ischemia in the rat. Neurosci Lett 2004; 357: 211–214.

17. Schmid H, Cohen CD, Henger A, Irrgang S, Schlondorff D, Kretzler M. Validation of endogenous controls for gene expression analysis in microdissected human renal biopsies. Kidney Int 2003; 64: 356–360.

18. Stan SD, Kar S, Stoner GD, Singh SV. Bioactive food components and cancer risk reduction. J Cell Biochem . 2008 May 1;104(1):339-56.

19. Wang LS, Stoner GD. Anthocyanins and their role in cancer prevention. Cancer Lett . 2008 Oct 8; 269(2):281-90.

20. Wang LS, Hecht SS, Carmella SG, Yu N, Larue B, Henry C, McIntyre C, Rocha C, Lechner JF, Stoner GD. Anthocyanins in black raspberries prevent esophageal tumors in rats. Cancer Prev Res (Phila Pa) . 2009 Jan; 2(1):84-93.

21. Holm, S. 1979. A Simple Sequentially Rejective Multiple Test Procedure. Scandinavian Journal of Statistics 6: 65-70.