THE EVOLUTION OF INBREEDING IN WESTERN REDCEDAR (THUJA PLICATA: CUPRESSACEAE) by LISA MARIE O'CONNELL B.A. University of Ottawa, 1993 B.Sc. Dalhousie University, 1995 M.Sc. Queen's University, 1997 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE STUDIES (Department of Forest Sciences) We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA 2003 © Lisa Marie O'Connell, 2003 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of forfs't Sci e rt c*5 The University of British Columbia Vancouver, Canada Date April H , 2^003 DE-6 (2/88) Abstract Long-lived woody plants usually show high levels of outcrossing, inbreeding depression and genetic diversity compared to other plants. A review of the literature showed a mean oucrossing rate of 83.5 in conifers, and a positive, but weak, correlation between outcrossing and genetic diversity. Among conifers, western redcedar (Thuja plicata, Cupressaceae) has one of the highest rates of self-fertilization and lowest amount of genetic diversity, and thus offers the opportunity to study the evolution of inbreeding in a predominantly outcrossing group of plants. This thesis links the evolution of inbreeding in redcedar with a loss in inbreeding depression and genetic diversity. Using one polymorphic isozyme marker, I obtained an average population outcrossing estimate of 71% over six natural populations of redcedar. I developed 13 highly polymorphic microsatellite markers to conduct a finer-scale study of the mating system and genetic structure of redcedar. A new method of bulking seedlings to estimate outcrossing rates was used to identify ecological correlates of outcrossing. Selfing rates increased significantly with tree height in four different populations. Pollen from larger trees probably made up a larger proportion of the surrounding pollen cloud, increasing self-pollination. There was no variation, however, in the amount of inbreeding among crown positions within trees. In a seed orchard, a combination of controlled crosses and isozyme markers showed evidence that post-pollination competition between embryos within an ovule decreased selfing. I used eight microsatellite loci to study patterns of range-wide genetic structure in redcedar. A phylogeographic analysis suggests that redcedar probably survived in three separate refugia during the last glaciation. These results also suggest that if a species-wide bottleneck is at the root of reduced genetic diversity in redcedar, it probably predates the last glaciation. The combination of an inbreeding mode of reproduction and a bottleneck probably contributed to the decrease in genetic diversity presently observed in redcedar. Finally, after screening 80 trees at eight microsatellite loci, a single stepwise mutation was observed, yielding a somatic mutation rate of 6.3 x 10"4 (95% CI: 3.0 x 10"5 - 4.0 x 10"3) mutations per locus per generation in western redcedar. iv Table of Contents Abstract ii Table of Contents iv List of Tables viii List of Figures x List of Appendices • X1i Acknowledgments XU1 Published papers X1V Chapter 1 General introduction and overview 1 The evolution of plant mating systems 1 Conifers 3 Patterns of genetic diversity in conifers 3 Review of outcrossing rates in conifers 6 Outcrossing rate and genetic diversity 8 Natural populations vs seed orchards 10 Inbreeding depression in conifers 11 Mating system of conifers 11 The genus Thuja (Cupressaceae) 12 Western Red Cedar (Thuja plicata) 12 Ecology of Thuja plicata - 12 Genetic diversity in Thuja plicata 13 Inbreeding depression in Thuja plicata 13 Species-wide bottleneck 14 Thesis overview 15 Chapter 2 The mating system in natural populations of western redcedar 16 Introduction 16 Materials and methods IV Sample collections IV Isozyme analyses 18 Data analysis 18 Results 19 Discussion 20 Outcrossing rates 20 Factors affecting mating systems 20 Correlation of paternity 22 V Chapter 3 Characterization of microsatellite loci in western redcedar 24 Introduction 24 Materials and methods 24 Clone development 24 Screening for polymorphisms 25 Results and Discussion 26 Chapter 4 Fine-scale estimation of outcrossing in western redcedar with microsatellite assay of bulked DNA 29 Introduction 29 Materials and methods 31 Bulking tests 31 Sample collections 32 DNA assay of bulks 33 Estimation of outcrossing from bulk samples 34 Results 36 Allele detection 36 Bulking tests 37 Genetic diversity 39 Outcrossing rates 39 Discussion 43 Variation in outcrossing rates 43 Population outcrossing rates 44 Bulking samples 45 Chapter 5 Polyembryony and early inbreeding depression in a self-fertile conifer, Thuja plicata (Cupressaceae) 46 Introduction 46 Materials and Methods 48 Pollinations 48 Seed viability 50 Embryo competition 51 Expected seed set and selfing with polyembryony 52 Fitness of self-pollen 54 Results 56 Seed set 56 Realized selfing rates 60 Success of self pollen 62 vi Discussion 63 Polyembryony as a rescue mechanism 63 Embryo competition 64 Early inbreeding depression self-incompatibility 65 Purging of inbreeding depression 65 The importance of pre-pollination mechanisms 66 Chapter 6 Range-wide genetic structure and diversity in western redcedar 68 Introduction 68 Matherials and methods 70 Sample collection 70 Microsatellite screening 74 Diversity analyses 74 Phylogeography 75 Clines in allele frequencies 76 Linkage disequilibrium 76 Bottleneck test 76 Results 78 Genetic Diversity 78 Genetic Structure 80 Isolation by distance 82 Northern vs southern populations 84 Mating system 85 Allele size distribution 85 Clines in allele frequencies 89 Bottleneck test 92 Discussion 95 Phylogeographic structure 95 Population differentiation 99 Reduction in genetic diversity 100 Timing a species-wide bottleneck 101 Inbreeding and genetic diversity 102 Chapter 7 Somatic mutations at microsatellite loci in western redcedar 104 Introduction 104 Materials and methods 105 Estimating mutation rate 105 Sample collections 107 vii Microsatellites 108 Results 109 Microsatellite mutations 109 Type of mutation 110 Somatic mutation rate estimate 110 Discussion Ul Somatic mutation rate Ul Mutation model 112 The consequences of somatic mutations in redcedar 112 Genetic mosaicism 113 Chapter 8 General discussion and conclusions 114 Main findings 114 Mating system 114 Inbreeding depression 115 Genetic structure and diversity 117 Tying it all together 118 Further research 119 Review of mating systems in trees 119 Glacial refugia 119 Allele distribution 120 Mating system at the edge of the distribution 121 Related species 121 References 123 Vlll List of Tables 1.1 Mean levels of within population isozyme variation in three species of Thuja compared to mean levels in gymnosperms and other plants 5 1.2 Mean outcrossing rate (t ± SD) by genus in 52 species of conifers 6 1.3 Outcrossing rates in 13 species of conifers with estimates from both natural and seed orchard populations 10 2.1 Locations, sample sizes (AO, gene frequency of the most common allele at locus G6pd, population outcrossing rates (0 and correlation of paternity (rp) of Thuja plicata populations in southwestern British Columbia 23 3.1 Characterization of Thuja plicata microsatellites in a coastal and two interior populations ... 27 4.1 Probabilities of band patterns observed for a single progeny, and for bulked progenies of sizes 2 and 3, conditioned upon maternal genotype (homozygous A,A, or heterozygous A,A,) 35 4.2 The total number of alleles detected in samples bulked before DNA extraction in two trees of Thuja plicata. Samples were scored at four microsatellite loci (TP1, TP3, TP9 and TP11) 38 4.3 Genetic diversity measures and total number of alleles detected at four microsatellite loci (TP1, TP3, TP9 and TP11) in four natural populations of Thuja plicata 40 4.4 Outcrossing rates (SE) at different positions within the crown of trees in four natural populations of Thuja plicata 41 4.5 Mean tree heights and individual tree outcrossing rates (0 in four populations of Thuja plicata 43 5.1 Experimental design of a pollination experiment in four trees in a Thuja plicata seed orchard. One hundred percent self-pollen (selfed) and 0% self-pollen (crossed) treatments as well as pollen mixtures with three different ratios of self/cross pollen (25%/75%; 50%/50%; 75%/25%) were applied to each tree 49 5.2 Probabilities of setting a full seed (/•) and setting a selfed seed (s,) with one, two or three embryos per ovule (n) 53 5.3 The proportion of full seeds (SE) and inbreeding depression at the seed stage in four western redcedar trees (181, 295, 431 and 432) 58 ix 5.4 The proportion of full seeds expected (/•) based on the number of embryos per ovule (n) and the proportion of self pollen applied (pk) in four Thuja plicata trees with varying levels of embryo viability 59 5.5 The proportion full seeds expected (j)) without polyembryony (when n = 1) and observed full seeds for three different proportions of self-pollen in four Thuja plicata trees 60 5.6 The proportion of selfed seeds expected (when n = 1) and observed for three different proportions of self-pollen in three Thuja plicata trees 62 5.7 Fitness of self-pollen relative to outcross-pollen (ws) when applied at different proportions in three Thuja plicata trees 63 6.1 Location of 23 sampled populations of Thuja plicata '. 73 6.2 Mean diversity at eight microsatellite loci in 23 populations of Thuja plicata 75 6.3 Genetic diversity measures in 23 populations of Thuja plicata at seven microsatellite loci..