In The Princeton Guide to Evolution ed. J. Losos. Princeton University Press, Princeton, New Jersey, pp. 356-362. IV.8 Evolution of Mating Systems: Outcrossing versus Selfing Spencer C. H. Barrett
OUTLINE that transitions to predominant selfing have occurred on numerous occasions, although selfing lineages are often 1. Definitions and measurement short-lived because of the negative effects of selfing on 2. Variation in mating patterns the genome. 3. Evolution of self-fertilization 4. Mechanisms of selection 5. The problem of mixed mating 6. Evolutionary history GLOSSARY Floral Design and Display. The morphological features of Mating systems vary enormously among groups of or- flowers and inflorescences that influence pollen dis- ganisms. This has led to diverse definitions and ap- persal and mating patterns in flowering plants; floral proaches for investigating their evolution and main- design involves characteristics of individual flowers, tenance. Animal mating systems are characterized by including their size, structure, color, and the spatial different patterns of parental investment in offspring and temporal presentation of female (pistil) and male and variation in the extent to which sexual selection (stamens) sex organs, and floral display concerns shapes male and female traits (see chapter VII.4). A the number of open flowers on a plant and their ar- primary focus of most studies is determining the causes rangement within and among inflorescences. and consequences of variation in mate number for fe- Inbreeding Depression. The reduction in viability and/ males and males. In contrast, in hermaphrodite organ- or fertility of inbred offspring in comparison with isms, particularly plants, the emphasis is largely on de- outbred offspring. It results primarily from the ex- termining the incidence of cross- and self-fertilization pression of deleterious recessive alleles in homo- and its fitness consequences. Most studies of mating- zygous genotypes and is expressed most strongly system evolution have been conducted on plants and when inbreeding occurs in primarily outcrossing largely concern the ecological and genetic mechanisms populations; a key parameter (d) in models of mat- responsible for evolutionary transitions from outcrossing ing-system evolution. to predominant self-fertilization, and the extent to which Mating System. Broadly defined, the mode of trans- mixed mating can be maintained as a stable strategy. mission of genes from one generation to the next Models of mating-system evolution involve the balance through sexual reproduction; however, in species between the transmission advantage of alleles affecting with separate sexes, it largely concerns the quantity the selfing rate and the reduced fitness of self-fertilized and quality of mates obtained by males and females. offspring because of inbreeding depression. Selfing pro- In contrast, in hermaphrodites, it is usually defined vides reproductive assurance whenever outcrossing is as the relative frequency of cross-fertilization (out- limited by the availability of pollinators or mates and crossing) and self-fertilization (selfing) in a popu- thus low density often plays an important role in mating- lation. In this article the last definition is used. system transitions. Reconstruction of the evolutionary Modes of Self-pollination. The various ways in which self- history of mating systems using phylogenies indicates pollination occurs in flowering plants, distinguished Evolution of Mating Systems 357
primarily by reproductive expenditure, whether a influences how and with whom a plant or animal can pollen vector is involved, and timing, relative to mate. But although the types of mating systems are cross-pollination. The evolution of selfing depends closely linked to the distinctive features of particular critically on the particular mode of self-pollination. taxa, it is not uncommon to find quite different mating Pollen and Seed Discounting. Important reproductive strategies among closely related species, especially in parameters for determining whether selfing will flowering plants. This indicates that mating systems are evolve in flowering plants; pollen discounting is evolutionarily labile and can respond to natural selec- the loss in outcrossed siring success caused by self- tion; therefore, understanding mating-system diversity pollination, whereas seed discounting is the reduc- has been a major theme in evolutionary biology since tion in outcrossed seed production caused by selfing, Charles Darwin’s influential work on the topic. either because selfing preempts ovules that could Reproductive biology is replete with terms for what have been outcrossed, or because self-fertilized are seemingly similar phenomena. Before we begin, it is seeds consume resources that could have been al- important to clarify what is meant by “mating system” located to outcrossed seeds. and how it relates to other terms associated with sexual Pollen Limitation. The reduction in potential seed pro- reproduction. This is important because the term has duction that occurs when ovules remain unfertilized various definitions and different usages, particularly in and too few embryos survive genetic death to com- the botanical versus zoological literature. The distinc- pete for maternal resources; results from insufficient tions depend largely on whether biologists are working delivery of pollen quantity and quality to flowers. on species with separate sexes (dioecy) versus those Persistent pollen limitation can result in the evolu- studying hermaphrodite (cosexual) organisms. Terms tion of selfing. that are commonly used in reproductive biology involve Reproductive Assurance. An increase in seed production the relative importance of sexual versus asexual re- caused by selfing when conditions for outcrossing production (reproductive system) (see chapter III.9), the are unfavorable because of an absence of pollinators occurrence of separate sexes versus hermaphroditism or mates; requires plants to be self-compatible and (sexual system), and within a population, who mates generally capable of autonomous self-pollination. with whom and how often (mating system) (see chapter Self-compatible and Self-incompatible. The two condi- VII.4); however, workers studying hermaphroditic or- tions that determine whether fertile hermaphrodites ganisms usually define mating system more specifically have the potential to produce offspring from self- as the relative frequency of cross-fertilization and self- fertilization. In plants, self-compatibility is the ability fertilization, and this is the definition used here, because to produce abundant seed following self-pollination, the main focus of this article is mating patterns in her- whereas in a self-incompatible plant, few, if any, seeds maphrodites. It is also important to note that the term are produced by self-pollination. Self-incompatibility breeding system is often used synonymously with mating is the most common antiselfing mechanism in flow- system in the animal and plant literature but usually ering plants. more broadly, to include mating behavior and parental care in animals, and the diverse reproductive traits that promote particular mating patterns in plants. Animal biologists in the field of behavioral ecology 1. DEFINITIONS AND MEASUREMENT classify mating systems largely on the basis of mate The mating system is a key life history trait because it number per male and female and are particularly inter- determines the quantity and genetic quality of offspring ested in determining the variance in reproductive success produced by an individual and thus the individual’s of the sexes and thus the scope of sexual selection. Here, reproductive fitness. One of the most striking features the mating system is viewed as a behavioral strategy for of organismal diversity is the existence of numerous obtaining mates, which also encompasses the nature of mating strategies, even though all serve the same basic parental care. Typical classes of animal mating systems function: to promote parental reproductive success. include monogamy, in which each sex mates with only Why do such diverse adaptations exist for an activity one partner during its lifetime, polygyny, where females crucial for persistence of species of sexual organisms? mate with a single partner but males are variable in mate Organismal variation in mating systems is determined, number, and polyandry, in which the reverse pattern in part, by the biological features that characterize a occurs. Information on the types of animal mating sys- particular taxon. For example, whether a group is sed- tems provides a powerful tool for predicting the degree entary or mobile and what type of sexual system it and direction of sexual dimorphism, as Darwin origi- possesses (e.g., hermaphroditism versus separate sexes) nally pointed out. These topics are discussed in more 358 Evolutionary Processes detail in Section VII: Evolution of Behavior, Society, 2. VARIATION IN MATING PATTERNS and Humans. Darwin wrote extensively about animal mating sys- The use of genetic markers for measuring mating pat- tems and sexual selection, but he also published three terns in populations has uncovered considerable varia- books on the reproductive biology of plants. This body of tion among plant species, and this diversity often pro- work initiated contemporary studies of mating-system vides the template for evolutionary transitions. Most evolution in plants. Because of the predominantly her- observed variation in outcrossing rate occurs in self- maphroditic sexual systems of plants, Darwin was par- compatible species in which environmental and demo- ticularly interested in how they avoid self-fertilization graphic factors can play an important role in determining and its harmful effects on progeny fitness (see chapter the amount of outcrossed pollen delivered to flowers by IV.6), which he demonstrated in numerous species pollen vectors (animals, wind, and water). Species with through controlled pollinations and comparisons of the strong self-incompatibility systems, or those that are performance of self- and cross-fertilized progeny. He highly selfing (autogamous), exhibit much less variation demonstrated that plants possess numerous structural in outcrossing rate; because of this, they have tended to contrivances (floral designs) that reduce the incidence of be underrepresented in mating-system surveys. Never- self-pollination and promote cross-pollination. Darwin theless, a picture is emerging of the distribution of out- also identified remarkable diversity in floral biology and crossing rates among seed plants, and this has stimulated mating systems among closely related species. This var- much interest and controversy because of its relevance to iation has enabled investigation of the ecological and theoretical models of mating-system evolution. genetic mechanisms responsible for evolutionary tran- Marker-based estimates of outcrossing rate from 345 sitions in mating systems. In the following sections we species representing 78 families by Goodwillie and col- will explore this topic further, focusing in particular on leagues (2005) demonstrated near-continuous variation plants, about which the most is known, but also drawing from highly selfing to obligately outcrossed species; parallels where possible with hermaphroditic animal however, among this sample, 42 percent of the species groups. could be classified as exhibiting mixed mating (a mixture Early efforts to understand mating patterns in her- of both outcrossing and selfing) in which t ranged from maphrodites depended on inferences from observations 0.2 to 0.8. As we will see later, the occurrence of mixed of behavior and the morphology of reproductive organs. mating presents an important challenge for theoretical For example, in plants, the relative positions of female models of mating-system evolution. A particularly and male sexual organs along with observations of pol- striking pattern is the finding that the distribution of linator foraging provided clues to the likely occurrence of outcrossing rate differs significantly between animal- cross- versus self-fertilization; however, such inferences pollinated (biotic) species and those pollinated by wind were often unreliable, and more importantly, they did or water (abiotic). Biotically pollinated species are al- not provide quantitative estimates of the types of mating most twice as likely to exhibit mixed mating compared events that occur. In the 1970s, this situation changed to those that are abiotically pollinated. Why this differ- following demonstration of the utility of genetic markers ence occurs is at present unclear but may be associated for measuring mating parameters in populations. Since with stronger selection in abiotically pollinated plants then, estimates of the proportion of offspring produced by for mechanisms that reduce the incidence of self-polli- selfing, s, or its complement, the outcrossing rate (t =1–s), nation owing to the huge quantities of pollen typically have been obtained for a wide variety of species, provid- produced, particularly in wind-pollinated species. Re- ing unprecedented insights into the extent of mating- gardless of the mechanisms responsible, this difference system variation. Estimates of t and s portray selfing and does point to the important influence of pollination outcrossing through female function and provide popu- systems on mating patterns. lation-level estimates of mating patterns. In addition, the There are much fewer data available on mating-sys- degree of biparental inbreeding and the extent to which tem variation in hermaphroditic animals. It has been es- progeny are full sibs can also be estimated. More recent timated that about 5 percent of animal species are her- developments using hypervariable genetic markers such maphroditic, although this value is much higher, about as microsatellites now enable measurements of mating one-third, if arthropods are excluded. Hermaphroditism patterns at the individual level and also estimates of out- occurs in the majority of animal phyla and is especially crossed mating success through paternal function. The common in Annelida (ringed worms), Cnidaria (partic- assessment of paternity and male siring success is im- ularly corals), Mollusca (particularly the gastropods: portant because it provides an opportunity to evaluate the snails and slugs), Nematoda (roundworms), and Play- importance of sexual selection in plant populations. helminthes (flatworms), where most mating-system work Evolution of Mating Systems 359
100 much about the demographic and genetic factors re- sponsible for the evolution of selfing.
80 3. EVOLUTION OF SELF-FERTILIZATION The evolution of predominant self-fertilization (autog- 60 amy) from high levels of outcrossing is the most fre- quent mating-system transition in hermaphroditic or- ganisms. No accurate estimates have been made of the total number of origins of selfing, but it is likely that in
Percentile 40 large groups, such as the angiosperms, this transition has occurred many hundreds of times. The evolution Animal of selfing from outcrossing has attracted the attention of 20 Plant evolutionary biologists because it has diverse biological consequences. First, the effects of selfing on relative fit- 0 ness through inbreeding depression are well established, 0 0.2 0.4 0.6 0.8 1 and as we shall see, these effects play an important role Selfing
Table 1. Average Gene Contribution in historically large populations with inbreeding de- pression greater than 0.5, and selection for selfing when Outcrosser Selfer demographic factors such as population bottlenecks re- duce inbreeding depression to less than 0.5, and partial Ovule parent 1 1 selfing purges deleterious recessive alleles; however, as Pollen parent 1 2 Total 2 3 we have seen, surveys of outcrossing rates in plants and animals indicate that many species exhibit mixed mat- ing. What accounts for this apparent discrepancy and is there evidence that mixed mating can be a stable mating also reduces the transmission advantage of a selfing var- strategy? iant. Current work is attempting to measure these three Several explanations help to reconcile the mismatch parameters in an effort to test models and predict the between theory and empirical evidence on the distribu- course of mating-system evolution. Measurements of in- tion of mating systems. First, existing estimates of the breeding depression are relatively straightforward, but distribution of outcrossing rates are likely biased in terms pollen and seed discounting are more challenging to es- of taxonomic representation (many estimates are con- timate, and the extent to which they influence selection on centrated in some well-studied taxa, e.g., pines, gum the selfing rate in plant populations is at present unclear. trees, and grasses) and, as discussed earlier, an under- The other general explanation for the evolution of representation of the two ends of the distribution, thus selfing involves the advantage of selfing individuals over inflating the frequency of species with mixed mating. outcrossing individuals when pollinators or mates are Second, theoretical models predict the equilibrium mat- scarce, such as commonly occurs under low density. This ing system in a population, and it is probable that some is referred to as the reproductive assurance hypothesis species are not at equilibrium when sampled and, given and was originally suggested by Charles Darwin and the sufficient time, could be driven to predominant selfing or German naturalist Hermann Mu¨ ller. The reproductive outcrossing. Third, in many animal-pollinated species, assurance hypothesis differs from Fisher’s automatic the selfing component of mixed mating is likely a non- selection hypothesis because it predicts increased seed adaptive cost that plants pay by having large floral dis- production in selfers compared to outcrossers and re- plays to attract pollinators. Geitonogamy occurs because quires that self-compatible plants have the ability to self- pollinators transfer pollen between flowers on a plant; pollinate autonomously in the absence of a pollinator. this provides little benefit to fitness because of strong in- Considerable biogeographical evidence indicates that breeding depression and pollen discounting. Fourth, selfing populations occupy range margins, ecologically where pollinators exhibit spatial and temporal fluctua- marginal sites, or areas with reduced pollinator densities tions in density, mixed mating can arise by modes of self- where outcrossers are absent, all circumstances predicted pollination that provide reproductive assurance. For ex- by the reproductive assurance hypothesis; however, sur- ample, delayed selfing arises when flowers self-pollinate prisingly few field studies have provided experimen- after opportunities from cross-pollination have passed, tal evidence in support of this hypothesis, despite the and models indicate that this will always be adaptive. widespread occurrence of pollen limitation of seed set in Finally, more elaborate theory, integrating more ecolog- animal-pollinated species. It seems likely that both auto- ical and genetic details, has been able to show stable matic selection and reproductive assurance will turn out mixed mating, albeit often under restricted conditions. to be involved in many transitions from outcrossing to Nevertheless, the adaptive significance of mixed mating selfing in flowering plants, with demographic context remains problematic for most species, and the searchfor a determining their relative importance. general explanation continues to be elusive, probably because the phenomenon has diverse causes.
5. THE PROBLEM OF MIXED MATING Most genetic models of mating-system evolution, based 6. EVOLUTIONARY HISTORY on the balance between the transmission advantage of The recent availability of phylogenetic information for selfing and the costs of inbreeding depression, result in groups with variation in mating systems has enabled populations in which either predominant selfing or the reconstruction of the evolutionary history of out- outcrossing is an alternative stable state. Thus, theory crossing and selfing, leading to several generalizations: predicts a bimodal distribution of outcrossing rates re- (1) the transition from outcrossing to predominant self- sulting from selection for the maintenance of outcrossing fertilization often occurs repeatedly within lineages 362 Evolutionary Processes and is usually unidirectional; (2) selfing species most FURTHER READING often occur at the tips of phylogenies and are therefore relatively young in age; (3) species-rich groups and Barrett, S.C.H. 2003. Mating strategies in flowering plants: higher taxa are usually composed of predominantly The outcrossing-selfing paradigm and beyond. Philo- outcrossing species. These findings support an early sophical Transactions of the Royal Society B 358: 991– view proposed by the plant evolutionist G. Ledyard 1004. A review of the distinctive features of plants that Stebbins that the evolution of selfing represents an influence mating, emphasizing the functional links among evolutionary dead end. Why should this be so, given flowers, inflorescences, and plant architecture. Barrett, S.C.H., and L. D. Harder. 1996. Ecology and evo- that selfing clearly has short-term advantages in some lution of plant mating. Trends in Ecology & Evolution 11: ecological contexts? 73–78. A concise summary of the functional dimensions High rates of self-fertilization are associated with neg- of plant mating drawing particular attention to the rela- ative genomic consequences that make it unlikely that tions between pollen dispersal and mating patterns. selfing species will persist over long timescales. These Charlesworth, D., and B. Charlesworth. 1987. Inbreeding effects include a reduction in the amount of adaptive depression and its evolutionary consequences. Annual geneticvariation in populations, limiting their capacity to Review of Ecology and Systematics 18: 237–268. The adapt to environmental change, low recombination and a classic review on the importance of inbreeding depression reduced efficacy of natural selection, lowering opportu- to mating-system evolution in plants and animals. nities for purging deleterious mutations as well as for Goodwillie, C., S. Kalisz, and C. G. Eckert. 2005. The evo- lutionary enigma of mixed mating systems in plants: Oc- fixing adaptive mutations. Thus, weak purifying selec- currence, theoretical explanations and empirical evidence. tion and/or the operation of Muller’s ratchet (the irre- Annual Review of Ecology and Systematics 36: 47–79. versible accumulation of deleterious mutations) should A major synthesis of empirical and theoretical work on lead over time to mutational decay and the ultimate ex- outcrossing rate variation in seed plants and its relevance tinction of selfing lineages compared with those that ha- to the problem of mixed mating. bitually outcross. Jarne, P., and Auld, J. R. 2006. Animals mix it up too: The Evidence supporting the ephemerality of selfing distribution of self-fertilization among hermaphroditic lineages comes from comparative work on the tomato animals. Evolution 60: 1816–1824. Parallels between family, Solanaceae, where the distribution of self-in- mating-system variation in hermaphroditic animals and compatibility (SI) and self-compatibility (SC), traits that plants. Lloyd, D. G. 1992. Self- and cross-fertilization in plants. II. directly influence outcrossing and selfing, respectively, The selection of self-fertilization. International Journal of are associated with contrasting rates of species diversi- Plant Sciences 153: 370–380. Theoretical models of the fication. Many independent transitions from SI to SC selection of selfing from outcrossing, including the key occur in this group, but significantly, SI is never regained, parameters: inbreeding depression, pollen and seed dis- a pattern commonly observed in other plant families. counting, and reproductive assurance. This raises the question of how SI can be maintained over Shuster, S. M., and M. J. Wade. 2003. Mating systems and long timescales if it is being continuously broken down strategies. Monographs in Behavior and Ecology. Prince- because of selection for selfing. The answer lies in the ton, NJ: Princeton University Press. A quantitative frame- higher extinction rates of SC lineages compared to SI work for investigating mating systems and sexual selec- lineages, counterbalancing the frequent loss of SI. Tran- tion in animals with separate sexes. Vallejo-Marı´n, M., M. E. Dorken, and S.C.H. Barrett. 2010. sitions from outcrossing to selfing are driven ultimately The ecological and evolutionary consequences of clonality by short-term ecological and genetic mechanisms in local for plant mating. Annual Review of Ecology and System- populations; however, they can also have long-term atics 41: 193–213. The ways in which plant clonal strategies macroevolutionary consequences for patterns of biodi- affect mating patterns and the ecological and evolutionary versity, because mating systems influence character evo- consequences of geitonogamous self-fertilization caused by lution and species diversification. large floral displays.