Heredity (2009) 102, 214–217 & 2009 Macmillan Publishers Limited All rights reserved 0018-067X/09 $32.00 www.nature.com/hdy ORIGINAL ARTICLE Dobzhansky–Muller incompatibilities and adaptation to a shared environment

RL Unckless and HA Orr Department of Biology, University of Rochester, Rochester, NY, USA

Natural selection might drive the evolution of postzygotic We show that the probability of evolving a DMI falls as reproductive isolation even when allopatric populations adapt selection coefficients among beneficial alleles become less to identical environments, an idea first suggested by Muller similar. The reason is that if one locus is under much (1942). Here, we analyze this scenario mathematically, stronger selection than the other, that locus is much more focusing on the evolution of a Dobzhansky–Muller incompat- likely to experience a substitution first in both populations. ibility (DMI) between populations. Our results identify a This precludes the development of a DMI, which requires potential problem with Muller’s scenario: adaptation to different substitutions in the two populations. identical environments can often involve substitution of the Heredity (2009) 102, 214–217; doi:10.1038/hdy.2008.129; same alleles, precluding formation of a hybrid incompatibility. published online 14 January 2009

Keywords: Dobzhansky–Muller model; hybrid sterility; reproductive isolation; speciation

There can now be little doubt that natural selection plays later cause incompatibilities in hybrids formed between an important role in the evolution of postzygotic the populations. Second, allopatric populations might reproductive isolation. Over the last decade or so, adapt to the same environment but in different ways considerable work has shown that positive natural genetically. As populations can respond to identical selection plays a part in the evolution of both extrinsic selection pressures in different ways, two populations (ecological inviability or behavioral sterility) and intrin- might still arrive at different genotypes. If so, their sic postzygotic (hybrid inviability or sterility) isolation hybrids might still suffer incompatible gene combina- (reviewed by Coyne and Orr, 2004). Although the former tions. This scenario was first described by Muller (1942). role is perhaps not surprising—it seems intuitively clear Here, we point out a potential problem with this that selection shapes divergence in characters of ecolo- identical environment scenario. The problem is that it is gical significance—the latter role was less expected. It harder than it might first seem for two populations to was not obvious that substitution at the genes causing adapt to an identical environment in different ways, intrinsic hybrid sterility or inviability is driven by thereby allowing the evolution of postzygotic isolation. positive natural selection. Nonetheless, molecular popu- The reason is that natural selection, although not lation genetic analysis of several genes known to cause deterministic, can often cause the substitution of the hybrid sterility or inviability, for example, OdsH, Nup96, same mutations in two populations. This can lower Lhr and Hmr, shows strong evidence of positive dramatically the chance of forming a hybrid incompat- Darwinian selection (reviewed by Coyne and Orr, 2004; ibility. This problem does not arise when incompatibil- Orr et al., 2007). It seems clear, then, that many of the ities result from adaptation to different environments. substitutions underlying genic (Dobzhansky–Muller) In this note, we calculate the probability of evolving a incompatibilities are driven by selection. (This selection DMI when natural selection drives adaptation to an was not, of course, selection for postzygotic isolation but identical environment. We consider the simplest scenario for some other character; this evolution pleiotropically in which allopatric populations evolve by substitution of gives rise to lowered fitness in hybrids.) recurrent new mutations. Natural selection might drive the evolution of Dobz- hansky–Muller incompatibilities (DMIs) in two ways. First, allopatric populations might adapt to their respec- The model tive—and different—environments and hybrid sterility We begin by considering two allopatric populations that or inviability might arise as a pleiotropic side effect of experience no gene flow. These populations begin with this divergent selection. One population, for example, identical genotypes (both are fixed for wild-type alleles might adapt to a dry environment and another to a wet. at all relevant loci) and experience identical environ- The genes underlying these ecological adaptations might ments. At some point in time this shared environment changes and the populations begin to adapt to the new, Correspondence: RL Unckless, Department of Biology, University of but still shared, environment. (Environments need not be Rochester, 485 Hutchison Hall, Rochester, NY 14627, USA. E-mail: [email protected] strictly identical; formally, our calculations assume only Received 29 August 2008; revised 26 November 2008; accepted 7 that those aspects of the environment that affect selection December 2008; published online 14 January 2009 coefficients at the loci we follow are the same.) Mutation Hybrid incompatibilities in a shared environment RL Unckless and HA Orr 215 is recurrent and evolution involves the substitution of genotypes, is one minus the probability that all a popu- aÀ1 new mutations. We assume that all mutations have lations arrive at the same genotype: PDM ¼ 1À1/2 . independent effects on fitness. Following Gillespie (1984, 1991), we consider the strong selection (Nsb1) and weak Selection: two loci mutation (Nm51) domain. Results involving weak selection and/or strong mutation (for example, NmB1) Now consider the evolution by natural selection. In the might well differ. special case in which sA ¼ sB, matters are again simple. Following Nei (1976), we can simplify our analysis by Because A1 and A2 are equally likely to fix first in a considering haploids. This captures the essence of our population, there is again a one-fourth chance both problem without requiring us to deal with unnecessary populations evolve to be A1B0 and a one-fourth chance details about the dominance of incompatibilities. In the both evolve to be A0B1; the probability that two populations case of a two-locus DMI, both populations initially have arrive at different genotypes, causing postzygotic isolation, 1 genotype A0B0. In the new environment, mutations A1 is thus again PDM ¼ 2, as noted by Gavrilets (2004). Finding and B1 are beneficial. But, when brought together, A1 and the probability of incompatibility with arbitrary sA and sB is B1 cause an in compatibility. Relative fitnesses among not so straightforward, however. genotypes are: To find this probability, we take advantage of several results obtained by Gillespie (1984, 1991). Gillespie A B 1 considered a large population in which beneficial 0 0 5 A1B0 1+sA mutations are sufficiently rare (Nu 1) to have indepen- A0B1 1+sB dent fates (that is, clonal interference does not occur) and A1B1 1Àt, in which beneficial mutations have definite fitness advantages (Nsb1). Standard population genetic theory where sA, sB, and t are all positive. shows that a new beneficial mutation has a probability of PE Note that, once A1 is fixed in a population, B1 cannot fixation of 2s, where the approximation assumes that be fixed as this would yield the unfit A1B1 genotype; s is fairly small (Haldane, 1927). Because most new similarly, once B1 is fixed, A1 cannot be fixed. We assume mutations are lost accidentally, it takes time before that rates of mutation to A1 and B1 are equal; back- recurrent mutation produces a copy of a beneficial allele mutation to A0 and B0 is not allowed. that is destined to sweep to fixation. Gillespie showed A DMI arises if one population evolves the A1B0 that the waiting time to the appearance of such a genotype and the other evolves the A0B1 genotype, mutation is geometrically distributed with a mean of allowing formation of unfit A1B1 hybrids if the popula- 1/(2Nus) generations (in diploids, 1/(4Nus)). For a A0B0 tions were to come into contact. A DMI cannot occur if population, the mean waiting time to fixation of A1 is both populations evolve the same genotype, either A1B0 1/(2NusA) generations and the mean waiting time to or A0B1. In this case, hybrids between the populations fixation of B1 is 1/(2NusB) generations. As each of these would retain a pure-population genotype and would be times is approximately exponentially distributed, the perfectly fit. probability that A1 fixes first in a population equals the We calculate the ultimate probability that two loci probability that an exponential random variable with a form a DMI, PDM, that is, the probability that populations mean of 1/(2NusA) is smaller than an exponential arrive at incompatible genotypes after an extended random variable with a mean of 1/(2NusB). Gillespie (essentially infinite) period of time; in effect, then, we shows that this probability is PA ¼ sA/(sA þ sB). Conver- assume that taxa remain allopatric for very long periods sely, the probability that B1 fixes before A1 is PB ¼ of time. Note also that the loci we follow are capable of sB/(sA þ sB). Note that these values are independent of causing a hybrid incompatibility if they evolve in the population size (so long as it is large). P P ¼ ‘right’ way in our populations. PDM represents, therefore, As A and B are identical in our a 2 populations, a conditional probability: given that the loci we follow the probability that one population becomes A1B0 and are capable of causing a hybrid incompatibility, what is the other A0B1, yielding a DMI, is the probability that they do so? 2sAsB PDM ¼ 2 ð1Þ ðÞsA þ sB The neutral case

Though mathematically trivial, it is worth first consider- This probability is maximized when sA ¼ sB and ing the case in which the evolution is neutral: sA ¼ sB ¼ 0. PDM ¼ 1/2. This maximum probability of incompatibility We will see that this scenario represents a limit in the is identical to the probability of incompatibility under case of evolving a hybrid incompatibility by natural neutrality. selection. We still assume that t40, that is, A1B1 is unfit. Equation (1) makes good sense. When sA and sB are Populations will ultimately either fix A1 or B1. Under equal, there is a good chance that one population will neutrality, there is a one-fourth chance that both substitute A1 and the other B1, allowing the formation of populations fix A1 (arriving at A1B0) and a one-fourth a hybrid incompatibility. But when sA and sB are very chance that both fix B1 (arriving at A0B1). The probability different, both populations are likely to substitute the of an incompatibility is one minus the sum of these same allele—the one with the greatest advantage— 1 probabilities, PDM ¼ 2. arriving at the same genotype. This precludes formation This logic is extended trivially to more than two of a hybrid incompatibility. When sA ¼ 0.05 and populations, each of which evolves independently. The sB ¼ 0.001, for instance, the probability of incompatibility probability of incompatibility, that is, that at least one decreases to only 3.8%. The above logic is extended pair of a allopatric populations arrives at incompatible trivially to the case of a allopatric populations,

Heredity Hybrid incompatibilities in a shared environment RL Unckless and HA Orr 216 yielding unfit hybrid genotype. A hybrid incompatibility will P ¼ 1 À ½Šs =ðs þ s Þ aÀ½Šs =ðs þ s Þ a thus form unless two populations fix the same geno- DM A A B B A B types, which occurs with probability 1/n. The probability of incompatibility is therefore PDM ¼ 1À1/n.Ifn ¼ 3 loci, Although we do not know the actual values of s for PDM ¼ 2/3. Given a allopatric populations, we get aÀ1 beneficial mutations in nature, a reasonable argument PDM ¼ 1À1/n . can be made that s is approximately exponentially When substitutions are driven by natural selection, the distributed. Gillespie (1984, 1991) and Orr (2003) pointed n-locus case is messy mathematically. We thus present out that, given a variety of distributions of fitnesses (for results for the three-locus (n ¼ 3) case only, which example, normal, gamma, exponential, logistic and so captures the biological point we wish to make. Alleles on), extreme value theory shows that the right-hand tail A1, B1 and C1 are beneficial when alone or in pairs; of these distributions declines exponentially. Because but any individual that carries all three alleles suffers wild-type alleles are typically highly fit, this tail a DMI and has fitness 1Àt. We assume that fitnesses corresponds approximately to the distribution of selec- are otherwise independent, that is, multiplicative: tion coefficients among beneficial mutations (see Gille- A1B0C0 has fitness 1 þ sA, whereas A1B1C0 has fitness spie and Orr for details). Although this argument is far (1 þ sA)(1 þ sB), and so on. from decisive (see Joyce et al., 2008), especially when Populations will ultimately fix two of the three considering mutations at different loci, an exponential mutations. Which two depends on the magnitudes of distribution of s at least represents a reasonable guess sA, sB and sC. For example, the probability, P(A1,B1), that a about biological reality. population ultimately fixes alleles A1 and B1 is the sum of We can thus ask: What is the average probability of two events: that A1 is fixed first and then B1 or that B1 is incompatibility between a pair of populations when sA fixed first and then A1. Using Gillespie’s arguments to and sB are drawn from a common exponential distribu- calculate the probabilities of these events, straight- tion? The answer is forward calculations showP that Z1 Z1 s s þ Ps= s 2sAsB 1 A B P E½P Š¼ l2eÀlðsAþsBÞds ds ¼ ; ð2Þ PðA1; B1Þ¼ ð4Þ DM 2 A B sAsB þ sC s ðsA þ sBÞ 3 0 0 where Ps is the product of the three selection coefficients where 1/l is the mean s among beneficial mutations. and Ss is their sum. P(A1,C1) and P(B1,C1) have the same In words, Equation (2) shows that pairs of populations form, with the obvious change of subscripts. evolve in different directions—yielding a DMI in As postzygotic isolation evolves unless both popula- hybrids—one-third of the time. Surprisingly, this result tions evolve the same genotype, the probability of is independent of mean selection coefficient, population incompatibility is size, and all other biological parameters. The finding is 2 2 2 closely connected to that in Orr (2005) for parallel PDM ¼ 1 À PðA1; B1Þ À PðA1; C1Þ À PðB1; C1Þ ð5Þ adaptation, who did not consider reproductive isolation and who took a different approach mathematically. Table 1 shows that, when selection coefficients among A similar calculation shows that, given a indepen- the three alleles are similar, evolution of a hybrid dently evolving populations, the expected probability of incompatibility is likely. In particular, when sA ¼ sB ¼ sC, incompatibility is the chance of isolation is maximized at PDM ¼ 2/3. Z1 Z1 Again, then, the maximum probability of incompatibility

sA 2 ÀlðÞsAþsB under natural selection equals that under neutrality. E½PDMŠ¼1 À 2 a l e dsAdsB ðsA þ sBÞ But when the selection coefficients are very different, 0 0 postzygotic isolation becomes much less likely. If, a À 1 for instance, s ¼ 0.001, s ¼ 0.03 and s ¼ 0.05, P ¼ ð3Þ A B C DM a þ 1 decreases to 7.7%. Finally, we can ask about the mean probability of When a ¼ 2, we recover Equation (2). As expected, incompatibility when selection coefficients involved in postzygotic isolation is more likely with more allopatric three-locus incompatibilities are drawn from an expo- populations. nential distribution. Although the integration required appears intractable, numerical work shows that Selection: complex incompatibilities A DMI need not involve only two loci. Instead, genetic Table 1 Probabilities of postzygotic isolation as a function of analyses of hybrid sterility and inviability have shown selection coefficients among beneficial mutations that complex incompatibilities, involving three or more loci, are common (Cabot et al., 1994; Orr, 1995). We can Number of loci Selection coefficients (sx) PDM extend our calculations to these cases. 2 Neutral 0.500 It is again worth considering the neutral case. Consider 0.01, 0.01 0.500 two allopatric populations that experience recurrent 0.005, 0.01 0.444 mutation at n loci with no back-mutation. Though each 0.001, 0.01 0.165 mutation is neutral, any individual who carries all n 3 Neutral 0.667 alleles suffers a genic incompatibility. Given enough 0.01, 0.01, 0.01 0.667 time, each population will ultimately fix nÀ1 of the 0.005, 0.075, 0.01 0.458 0.001, 0.075, 0.01 0.168 alleles; the last allele cannot fix as this would yield the

Heredity Hybrid incompatibilities in a shared environment RL Unckless and HA Orr 217 E E[PDM] 3/7. The mean probability of incompatibility is subtle. Our model assumes that selection coefficients thus increases somewhat when DMIs are complex. among beneficial mutations are independent. Thus, if one mutation fixes, selection coefficients at all other Conclusions mutations remain unchanged. This need not, of course, be true and it not generally true under genetic conflict. Although the mathematics involved in our analysis is With conflict, a mutation might become beneficial only simple, the biological implications that emerge seem after another has fixed. (For example, an allele that non-trivial. We have obtained two main results. First, suppresses meiotic drive is favored only after a driver when allopatric populations adapt to identical environ- mutation has appeared.). Put differently, genetic conflict ments and the loci we follow can potentially form a DMI, does, in fact, involve a kind of adaptation to different the probability that they do so is constrained in a way environments, but the relevant ‘environments’ are that does not occur when populations adapt to different genetic, that is, involve genotypes at other loci. environments. The reason is that the evolution of a DMI requires that populations evolve different genotypes. But adaptation to an identical environment is somewhat Acknowledgements repeatable: populations will often substitute the same We thank D Presgraves, M Turelli, the members of the alleles, arriving at the same genotype. (See Wood et al. (2005), who reviewed the evidence for parallel genetic Orr laboratory and three anonymous reviewers for helpful discussion and/or comments. This study was evolution in the quantitative genetic and experimental evolution literature). When populations independently supported by funds from the NIH (GM51932) to HAO and from the Robert and Mary Sproull Fellowship of the arrive at the same genotype, a DMI cannot arise. Of course, if selection is spatially heterogeneous, that is, University of Rochester to RLU. populations adapt to different environments, incompat- ibilities can arise more easily (see also Navarro and References Barton, 2003; Kondrashov, 2003; Gavrilets, 2004). Our analysis further shows that, as the number of loci Cabot EL, Davis AW, Johnson NA, Wu CI (1994). Genetics of involved in the incompatibility increases, the probability reproductive isolation in the Drosophila simulans clade: of evolving an incompatibility also increases. complex epistasis underlying hybrid male sterility. Genetics Our second finding is that the probability of post- 137: 175–189. zygotic isolation decreases as selection coefficients Coyne JA, Orr HA (2004). Speciation. Sinauer Associates Inc.: among beneficial mutations become less similar. In fact, Sunderland, MA. Gavrilets S (2003). Models of speciation: What have we learned the maximum probability of isolation occurs when in 40 years? Evolution 57: 2197–2215. selection coefficients at each locus are equal; in this case, Gavrilets S (2004). Fitness Landscapes and the Origin of Species. the probability of incompatibility equals that under Princeton University Press: Princeton, New Jersey. neutrality. Under both the equal-s and neutral scenarios, Gillespie JH (1984). Molecular evolution over the mutational evolution randomly chooses alleles at the relevant loci landscape. Evolution 38: 1116–1129. for substitution. Populations thus often arrive at different Gillespie JH (1991). The Causes of Molecular Evolution. Oxford genotypes, allowing postzygotic isolation. It should be University Press: New York. noted, however, that the time-scales for the evolution of Haldane JBS (1927). A mathematical theory of natural and postzygotic isolation under the equal-s and neutral artificial selection, part V: selection and mutation. Proc Camb scenarios are very different (Gavrilets, 2004): neutral Phil Soc 28: 838–844. Joyce P, Rokyta DR, Beisel CJ, Orr HA (2008). A general extreme evolution is slow. value theory model for the adaptation of DNA sequences Most of our mathematical results have not been noted under strong selection and weak mutation. Genetics 180: before and the reason seems clear. Many previous 1627–1643. analyses of the Dobzhansky–Muller model (including Kondrashov AS (2003). Accumulation of Dobzhansky–Muller our own, for example, Orr, 1995) typically focused on the incompatibilities within a spatially structured population. long-term accumulation of alleles causing hybrid in- Evolution 57: 151–153. compatibilities and were agnostic about whether sub- Muller HJ (1942). Isolation mechanisms, evolution and tem- stitutions were driven by natural selection or genetic perature. Biol Symp 6: 71–125. drift. Selection coefficients among beneficial alleles did Navarro A, Barton NH (2003). Accumulating postzygotic isolation genes in parapatry: a new twist on chromosomal not, therefore, generally appear in previous theory (for speciation. Evolution 57: 447–459. important exceptions, see Nei (1976); Gavrilets (2003, Nei M (1976). Mathematical models of speciation and genetic 2004); Gavrilets, in particular, produced theory that distance. In: Karlin S, Nevo E (eds). Population genetics and considered the role of selection in the buildup of ecology. Academic Press Inc.: New York. incompatibilities in spatially-structured populations). Orr HA (1995). The population genetics of speciation: In closing, it is important to guard against a possible the evolution of hybrid incompatibilities. Genetics 139: misinterpretation of our results. We do not suggest that 1805–1813. the evolution of postzygotic isolation in identical Orr HA (2003). The distribution of fitness effects among ecological environments is unlikely. Indeed, a common beneficial mutations. Genetics 163: 1519–1526. cause of reproductive isolation by DMIs could be genetic Orr HA (2005). The probability of parallel evolution. Evolution 59: 216–220. conflict, in which substitutions are driven by arm races Orr HA, Masly JP, Phadnis N (2007). Speciation in Drosophila: among different sites in the genome. Such genetic- from phenotypes to molecules. J Hered 98: 103–110. conflict-based reproductive isolation can obviously Wood TE, Burke JM, Rieseberg LH (2005). Parallel genotypic evolve in identical ecological environments. The reason evolution: when evolution repeats itself. Genetica 123: our results do not militate against this form of speciation 157–170.

Heredity