Effects of Mutation on Selection Limits in Finite Populations with Multiple Alleles

Effects of Mutation on Selection Limits in Finite Populations With Multiple Alleles

Zhao-Bang Zeng, Hidenori Tachida andC. Clark Cockerham Department of Statistics, North Carolina State University, Raleigh, North Carolina 27695-8203 Manuscript received March 10, 1989 Accepted for publication May, 3, 1989

ABSTRACT The ultimate response to directional selection (ie.,the selection limit) under recurrent mutation is analyzed by a diffusion approximation for a population in which there are k possible allelesat a locus. The limit mainly depends on two scaled parameters S (= 4Nsa,) and 0 (= 4Nu) and k, the number of alleles, where N is the effective population size, u is the mutation rate, s is the selection coefficient, and a: is the variance of allelic effects. When the selection pressure is weak (S 5 0.5), the limit is given approximately by 2SaJ 1 - (1 + c‘)/k]/(e + 1) for additive effects of alleles, where c is the coefficient of variation of the mutation rates among alleles. For strong selection, other approximations are devised to analyze the limit in different parameter regions. The effect of mutation on selection limits largely relies on the potential of mutation to introduce new and better alleles into the population. This effect is, however, bounded under the present model. Unequal mutation rates among alleles tend to reduce the selection limit, and can have a substantial effect only for small numbers of alleles and weak selection. The selection limit decreases as the mutation rate increases.

HE cause of responses to directional selection ROBERTSON(1 955, 1964) for estimating the amount T with recurrent mutation can be generally di- of new genetic variance.This mutationmodel assumes vided into two phases. In the first phase, the response that the genetic variance produced by mutation per to selection is rapid andmostly due togenetic variance generation is constant over time independent of the existing in the base population. As the genes in the background level ofpopulation mean andgenetic initial population approach fixation, gradually more variance. Many recent theoretical analyses used this response to selection comes from new genetic varia- mutationmodel, which includeKIMURA (1965), tion introduced by mutation and the process of re- LANDE(1975) and TURELLI(1 984) on the mainte- sponse is transformed into the second phase. Com- nance of genetic variance with mutation and stabiliz- pared with the first phase, the second phase of selec- ing selection; LANDE(1 976, 1977) and TURELLI,GIL- tionresponse, largely relying on new genetic LESPIE and LANDE(1 988) on the prediction andesti- variation,covers a much longer time scale with a mationof therate ofevolution of quantitative relatively small response per generation. But, sooner characters from genetic drift; CHAKRABORTYNEI and or later, the response will reach a plateau balancedby (1982) and LYNCH and HILL (1986) on the genetic mutation and directional selection because the poten- variation within and between populations; and HILL tial genetic variance accessible by mutation in a locus (1 982a,b) and HILL RASBASHand (1986) on long-term must be finite. In this paper, we formulate this equi- response to artificial selection. This model has the librium and analyze in particular thepopulation mean appeal of relating various evolutionary quantities to at this mutation-selectionlimit. The analysis is directly the parameter Vm, the rate of input of new genetic related to the evaluation of mutation effects on the variance by polygenic mutation, which is estimable long-term response of populations to artificial selec- from divergence andselection experiments under cer- tion. It has also some relevance to evolutionary prob- tain assumptions (LANDE 1975; HILL 1982b; LYNCH lems, such as selection for fitness in natural popula- 1988). But it also has some limitations. Since the tions. However, due to the nature of the problem, model does not produce a bound of genetic variance this kind of analysis depends very much on the muta- in an infinite population in the absence of selection,it tion model. is subject to the restrictions that the genetic variance Two extremepolygenic mutation models have been in a population is not near saturation (LANDE 1975) used in recent theoretical analyses. One is the constant and the populationmean is not very farfrom the variance model, originally proposed by CLAYTONand original value, because in reality there must be a limit to the range of allelic effects and hence the genetic The publication costs of this article were partly defrayed by the payment of page charges. This article must therefore be hereby marked“advertisement” variance at a locus. The observations that thebetween- in accordance with 18 U.S.C. $1734 solelyto indicate this fact. population genetic variance (CHAKRABORTYand NEI

Genetics 122: 977-984 (August, 1989) 978 Z.-B. C.Zeng, C. H. Tachidaand Cockerham 1982; LYNCHand HILL 1986) andthe cumulative ing discussion we analyze only the response from one selection response frommutation (HILL 1982b) locus with the understanding that a summation over asymptotically increase linearly with timewithout all lociis implied provided thatwe are willing to ignore bound by using this mutation model reflectthese the effects of linkage disequilibrium. While linkage limitations. disequilibrium builds up in transient states with direc- The other model is the k-altele model in which tional selection, the influence may be minor at equi- mutation is assumed to occur amongK alleles. In many librium. papers k is restricted to two or three or five and in others k is set to be arbitrary, even to be infinite. MODEL AND SELECTIONLIMITS These papers includeLATTER (1 960), BULMER(1 972), TURELLI(1 984), BARTON(1 986) and SLATKIN(1 987) Consider the conventional diffusion model of mul- on themaintenance of genetic variance with mutation tiple-allele selection with mutation in a finite popula- and stabilizing selection; COCKERHAM(1 984) and tion of effective size N. Assume thatthere are k COCKERHAMand TACHIDA(1 987) on the genetic var- possible alleles at a locus available for mutation in a iation within and between populations; KIMURA population. Let xi and ai be the frequency and phe- (198 1) on the molecular evolution under stabilizing notypic effect respectively of the ith allele, Ai, and let selection; and LI (1977) and WATTERSON(1977) on w = m(xl, . . . , xk-1) denote the mean fitness of the the heterozygosity under mutation and selection. In population, assumed to be approximatelyconstant, contrast with the constant variance model, the k-allele but varying by amounts of order O(l/N), due to the model behaves properly in differenttime horizons effect of small selective forces, as allele frequencies (COCKERHAM and TACHiDA 1987) and depends much vary. Foradditive effects of genes and directional more on the parameter 4Nu than V,, where N is the selection, iV = I + 2slxl + . . . + 2skXk where st is the effective population size and u is the mutation rate at selection coefficient of the ith allele. (For example, in a locus. In some problems, the results do not critically the case of phenotypic truncation selection si i(ut - depend on which mutation model is used (e.g., TUR- ii)/up where i is the intensity of selection, up is the phenotypic standard deviation of the character, and ELLI 1984). But in some other problems (the problems ii involving long-term response to drift and selection), is the mean effect of the alleles.) Suppose that in each generation A, mutates to A, the results differ markedly for different models. By with probability # j). Then, subject to the usual using the k-allele model, COCKERHAMand TACHIDA uji(i multinomial-type sampling variation from one gener- (1987) observed that without selection the between- ation to the next, the change, 6x,, in the ith allele’s population genetic variance does not increase indefi- relative frequency in one generation has a mean nitely, but to a finite equilibrium value, which is in sharp contrastwith the analyses of CHAKRABORTYand Mi 2 xpji - xi 2 uq NEI (1982) and LYNCH and HILL(1986). Despite this j .i difference, however, the two models give comparable results as to the predictions of the short-term change of genetic variances within and between very small populations starting from afixed population. a variance V,i = x,( 1 - xi)/2N, and 6x;, 6xj have a It is not possible to distinguish between the two covariance Vq = - xixj/2N. If we denote polygenic mutation models from experimental results, @(x*,. . . , xk-1; t) as the joint probability density of because the two models do not differ markedly on the the first k - 1 allele frequencies at generation t, the short-term accumulation of genetic variance from mu- Kolmogorov forward equation is tation and it is also very difficult to experimentally k-I k-1 k- 1 eliminate the confounding effects from selection and d9/dt = (G) 2 2 d2( Vij@)/dXIdX, - 2 d(Mi@)/dX,. drift. Most mutation accumulation and selection ex- :=I ;=I i= 1 periments are carried out on unselected base popula- Unfortunately no equilibrium solution seems to have tions for a relatively short time, yet the results are been obtained underthis general condition. However, usually ambiguous as to thepattern of effects of for the special case where ujz = ui for all j # i, the polygenic mutation (see CLAYTONand ROBERTSON above equation has a stationary solution as was first 1964; HOLLINGDALEand BARKER 197 1 ; MUKAI1979). shown by WRIGHT(1949) In this paper, we determine thepopulation mean at the equilibrium between mutation and directional selection by using the k-allele model. This equilibrium mean is the limit to selection under recurrent mutation. (Note that no limit is expected under the con- where ti = 4Nui, xk = 1 - x1 - . . . - &-I, and c is a stant variance model, see HILL 1982b). In the follow- normalizing constant. If @ is close to one so that the Mutation Effects on Selection Limits 979 w in the denominatorof (1) can be replacedwith one, where E denotes the expectation with respect to a. the stationary solution is If the initial population is maintained at equilibrium k with respect to mutation in the absence of selection in @ = c exp(2~W}n x?". (3) which the population mean is zero, (4) represents the i= 1 ultimate response to selection (the selection limit) that In the past several years a great deal of work (e.g., could be obtained from that initial population with LI 1977; WATTERSON1977) has been published using mutation. This formula is general for the values of (2) or (3) as a starting pointto discuss the maintenance parametersappropriate for adiffusion approxima- of genetic variance and the number of alleles in nat- tion, i.e., the scaled parameters 6 and S are of order ural populations. In this paper, we use (2) and (3) as a one. But since no further simplification can be made, starting point to discuss the properties of selection the numerical calculation is restricted to the cases of limits. small N and k. When N and k becomelarge, the The differencebetween (2) and (3) is negligible computation becomes intractable because there are when w is close to one. The advantage of using (2) is too many possible alternatives of ni to be considered. that the normalizing constant C can be formulated as For this reasonwe consider thefollowing two approx- a finite series. This facilitates numerical evaluation of imations. the population mean of the character at equilibrium, i.e., the selection limit. But the direct evaluation is WEAK SELECTIONAPPROXIMATION restricted to the cases of small numbers of alleles, k, and small population size, N, (see below). Some ap- The case of weak selection is special. Forthat, proximations of the selection limit can be made on(3) considerable analyses can be madeon the momentsof for some parameter regions of scaled parameters 0 = gene frequencies. When S is small (

+ ~ 1) ,=I The expected gene frequency at equilibriumis then e e(e + ( ) Assuming that theeffects of alleles are independent g(xjI al, . . . , ah) = s . . . l xj9dxl . . . dxk-1 of each other andalso independent of mutation rates, the selection limit is thus given by 2N =c C (2N)!(2s)" n-0 (2N - n)!r(O+ n + 1) R = 2% I: g(xi1 al,. . . , at)a, (ki= 1 ) (6) k apr(ci + ni) ,, C (fj + nj) k n i=II 1 ni! 2au: 1 - 2 t'/e2 = -e+ 1 ;=I where S denotes the expectation with respect to x. ( ) The population mean of the character atequilibrium for any distribution ofa with mean zero andvariance is thus given by 6:. This limit is proportionalto 2u31 - xi $"e'], k which is the equilibrium additive genetic variance in R = 2% C %(xj 1 al,.. . ,ak)aj the absenceof selection in an infinite population L1 ) (COCKERHAMand TACHIDA1987). For equal mutation rates among alleles, i.e., ui = uj for all i andj, Xi t?/fI' =2% I 2N (2N)!(2s)" CC (4) = l/k. If the mutation rates are different,E~tP/6' = (1 1 n=O (2~- n)!r(e + n + 1) + c2)/k, where c is the coefficient of variation of the k a:'r(t, + ni>l mutation rates. Thus thelimit increases as k increases c ( i (€1 + nj)aj) II n J=l i= 1 n,! f to a maximum whenk = co and decreases asc2 increases 980 Z.-B. Zeng, H. TachidaandCockerham C. C. if k is finite. COCKERHAMand TACHIDA(1987) point butions of a. For example, if a is normally distributed out that this latter effect can be substantial for few with mean zero and variance a: [denoted a - alleles. The limit is linearly related to ./(e + l), increasing linearly with CY and decreasing with B at the rate (0 + l)-'. However, as shown later, this relation R,v= ~cYu,~. (9) does not hold well when S is large (roughly S > 0.5). For a doublegamma distribution reflected about zero with density function f(a) = I/2[X/(7r I a I )]'I2 WEAK MUTATIONAPPROXIMATION exp(-X I a I ) where X = h/2aa [denoted a - Y(O,a,2)] (HILL 1982b), thelimit is Another method to approximate theselection limit is to assume that mutation is very weak. As B 4 0, the (X + a)S/* - (X - 43'2 population will be mostly monomorphic at equilib- R9= (X - .)(X + a)(@- a)1/2 + (X + a)q rium. This is assured by the assumption of additivity of gene effects (KIMURA 1956). When S is large (-. when I CY I < X or ISI (= I CY I a,) < m;otherwise m), the population will be mostly monomorphic for RYis divergent. This property of the double gamma the best allele available for mutation. For moderate distribution is peculiar. Itis so because the distribution and small S, thechance to be nearly fixed in the defined above has much longer tails than the normal population is, however, distributed amongalleles, and distribution. If a is uniformly distributed with mean the probability of the population being monomorphic zero and variance af [denoted a - %(O,U,~)], we have for allele Ai at equilibrium is approximately proportional to exp(aai),since Prob(monomorphic for A ) When S is small, all these limits converge to (6) for k = 2 x1 2 2 2 Prob(1 1 - 1/2N, 1/2N xi 0, i #1) --., and B --.) 0. Re is necessarily bounded. R,vis bounded exceptwhen S = a~or -to; but Ryis bounded - J' . . . Jb'llzN c exp-1 a xia, 1. only when I S I < &/2. In any case lRvl 1 I RA 2 1-1/2N I"'* 1 i=l j I Re1 for k +-00. k When the numberof alleles is finite, we can numer- n x:-] dxr-1 . . . dx1 ically evaluate (7) for different distributionsof a. i= 1 NUMERICALANALYSIS 2: c exp(aalj J'1-1/2N J"2*.. . The limit given by (7) is the maximum limit for 1/2N k given k, a and ai's, since, as 0 increases, the limit n x:-'dxk-I . . . dx1. decreases as shown by (6). So we begin with the I i=l numerical evaluation of (7) for random effects of ai's. Thus We simulated two kinds of distributions of a: normal distribution anddouble gamma distribution,both Prob(monomorphic for A,) with mean zero and variance one. Both distributions are symmetrical. We used the same definition of dou- = exp(aajjexp{aaj). ble gamma distribution as HILL (1982b),which gives /"j= 1 a highly leptokurtic distribution forallelic effects. The From this expression we expect the selection limit to value of R is evaluated by repeatedly sampling values be of ai's from the given distributions for various values of k and S. The results of numerical calculations are shown in Table 1 for the normal distribution and in Table 2 forthe double gamma distribution. Each value in the tables (except those for k = m in Tables 1 where B + 0. and 2 and S + UJ in Table 1) is the average of 10,000 This is an expectation of a ratio. When k is finite, replicate samples, f standard error. The values of R there is no simple expression for the limit for a given for S = m in Table 1 are the expected largest normal distribution of a. However as k 4 m, we can use the order statistics, and in Table 2 are theaverages of the law of large numbers for both numerator and denom- largest values of a in the samples. It is seen that R inator, that is increases with S and k and is bounded in both direc- tions (except in the case of S + a~and = m) in Table R = 2k$$ae""]/1$7,[ea"]. (8) k 1. The values of R for the doublegamma distribution With (8) we can derive the limit for different distri- are less than or equal to those for the normal distri- Mutation Effects on Selection Limits 98 1

TABLE 1 Selection limits(R 2 standard error) numerically evaluated from(7) for 0 << 1 and a - /Y(O,l)

k s = 0.1 S = 0.5 s = 1.0 s = 2.0 S = 5.0 S” 2 0.098 2 f 0.014 0.436 f 0.015 0.750 f 0.016 0.977 f 0.017 1.085 f 0.0171.13 3 0.129 f 0.012 0.621 f 0.013 1.048 f 0.014 1.429 f 0.015 1.642 f 0.0151.69 5 0.162 5 f 0.009 0.763 f 0.010 1.335 ? 0.0 12 1.912 f 0.014 2.254 f 0.014 2.33 10 0.187 f 0.006 0.871 f 0.008 1.608 f 0.010 2.451 f 0.013 2.959 f 0.012 3.08 20 0.187 f 0.005 0.936 & 0.005 1.779 f 0.008 2.867 f 0.01 1 3.584 +- 0.01 1 3.73 50 0.192 f 0.003 0.974 f 0.003 1.907 f 0.004 3.268 f 0.010 4.297 f 0.0 10 4.50 100 0.195 f 0.002 0.987 f 0.002 1.954 f 0.004 3.378 f 0.009 4.777 f 0.010 5.02 m 0.2 1 .o 2.0 4.0 10.0 m

TABLE 2 Selection limits(R f standard error) numerically evaluated from(7) for 0 << 1 and a - Y(0,l)(the doublegamma distribution)

k s = 0.1 S = 0.5 s = 1.0 s = 2.0 S = 5.0 S” 2 0.103 2 f 0.015 0.430 +- 0.017 0.620 f 0.018 0.776 f 0.018 0.908 f 0.018 1.043 f 0.016 3 0.137 3 & 0.012 0.586 f 0.016 0.898 f 0.019 1.160 f 0.019 1.367 f 0.019 1.430 f 0.018 5 0.162 f 0.010 0.762 f 0.016 1.244 f 0.020 1.676 f 0.022 1.987 f 0.021 2.041 f 0.020 10 0.187 f 0.007 0.968 f 0.015 1.743 f 0.023 2.554 f 0.025 2.997 ? 0.023 3.113 f 0.023 20 0.192 f 0.005 1.109 f 0.014 2.304 f 0.025 3.582 f 0.028 4.200 f 0.025 4.359 f 0.025 50 0.199 f 0.003 1.260 f 0.013 3.088 f 0.028 5.148 f 0.031 5.963 f 0.026 6.095 f 0.026 100 0.199 f 0.002 1.357 f 0.012 3.747 f 0.0306.509 f 0.032 7.307 f 0.0277.466 f 0.027 m 0.203 1.551 m m CC m

bution when k < 10; otherwise they are larger. The the limit is less proportionally reduced. This is illus- values of R for S = 0.1 in both tables agree with each trated in Table 4 as an example with €1 = 0.80, €2 = other very well. Note that forS = 0.1 (weak selection), 0.15, and t3 = 0.05 for three alleles (8 = 1 and c2 = the m-allele value multiplied by (k - l)/k works well 1). Comparing Table 4 with Table 3 (k = 3 and 6 = (in this case c * 0). However, when S 2 1,the 1) shows that the proportion of reduction of limit argument is not satisfactory, as expected. decreases from about 0.5 when S = 0.1 and 0.5 as To show the effect of 6 on R, we numerically expected to about 0.03 when S = 5. Thus c2/(k - 1) evaluated (4)for various values of 6 and S and equal is the conservative estimate of proportional reduction. mutation rates among alleles with k = 2, 3 and 5, N = When the number of alleles is not small and selection 20, and a - M(0,l). We use N = 20 here for the is not weak, the effect of unequal mutation on selec- tractability of numerical computation. The change of tion limits is small and can be ignored. 6 and S is achieved by changing u and s for fixed N and uz. Since in the diffusion equation method what DISCUSSION matters are8 and S rather than individual parameters Recently, HILL (1982a,b) and HILL and RASBASH N, u and s, the effect of N on the limit is negligible (1986) extensively analyzed the mutationeffect on the for given 6 and S. The results are shown in Table 3. response to artificial selection with the constant mu- The results are the averages of 10,000 replicate sam- tation variance model on a time scale observable in an plings for k = 2, 5000 for k = 3, and 1000 for k = 5. artificial selection experiment and concluded that mu- For fixed number of alleles, an increase in 6 decreases tations in the broadsense occurring during the exper- the selection limit, and the rate ofdecrease is satisfac- iment from whatever sources could be important to torily predicted by (6 + 1)” when S 5 0.5 approxi- long-term response to selection. However, due to the mately. For S larger, (8 + l)-’ overestimates the rate. mutation model used, their prediction of mutation When S = 5 and 6 = 5 (for k = 2, 3, and 5), the limits effect is notappropriate for an evolutionary time are only reduced by about a half of those for 6 + 0. scale. Roughly, the weak mutationapproximation works Here we discuss the eventual effect of mutation on well for 6 < 0.1. selection response. By using Wright’s formula of sta- The effect of unequal mutation rates among alleles tionary distribution of alleles, we have obtained the is to reduce theselection limit. When selection is weak, selection limit under recurrent mutation. For weak the limit is reduced by aproportion of c2/(k - 1) selection (roughly S I0.5) the limit is 8Nsa,2[ 1 - (1 + compared with that of equal mutations. This propor- c2)/k]/(6 + 1) approximately. It is then of interest to tion decreasesas k increases. When selection is strong, compare this limit with that without mutationto assess 982 Z.-B. Zeng, H. TachidaandCockerham C. C.

TABLE 3 Selection limits (Rf standard error) numerically evaluated from (4) for equal mutation rates with R = 2,3, and 5 [N= 20, a - Y(O,l)]

8 s = 0.1 S = 0.5 s = 1.0 s = 2.0 S = 5.0

k=2 0.098 f 0.014” 0.436 f 0.015” 0.750 f 0.016“ 0.977 f 0.017” 1.085 f 0.017” 0.01 0.1050.01 f 0.014 0.434 f 0.015 0.707 f 0.016 0.978 f 0.017 1.108 f 0.017 0.1 0.090 f 0.014 0.431 f 0.015 0.664 f 0.016 0.918 f 0.017 1.060 f 0.017 0.5 0.0620.5 f 0.014 0.359 f 0.0 15 0.529 f 0.015 0.809 f 0.016 0.969 f 0.016 1.o 0.045 f 0.014 0.228 f 0.015 0.414 f 0.015 0.689 f 0.016 0.930 f 0.016 2.0 0.034 f 0.014 0.180 f 0.014 0.296 f 0.015 0.513 f 0.014 0.776 f 0.016 5.0 0.012 f 0.014 0.124 f 0.014 0.155 +. 0.014 0.312 f 0.015 0.532 f 0.015 k=3 0.129 f 0.012b 0.621 f 0.013’ 1.048 f 0.014’ 1.429 f 0.015’ 1.642 f 0.01 5’ 0.01 0.126 f 0.017 0.635 f 0.018 1.010 f 0.021 1.421 f 0.022 1.635 f 0.021 0.1 0.125 f 0.016 0.590 f 0.018 0.954 f 0.020 1.366 f 0.021 1.615 f 0.021 0.5 0.107 f 0.016 0.445 f 0.017 0.783 f 0.019 1.173 f 0.020 1.509 f 0.021 1 .o 0.064 f 0.016 0.349 f 0.017 0.588 f 0.018 0.987 f 0.020 1.305 f 0.021 2.0 0.036 f 0.016 0.208 f 0.017 0.405 f 0.017 0.713 f 0.019 1.145 f 0.020 5.0 0.035 f 0.016 0.138 f 0.017 0.217 f 0.017 0.434 f 0.01 7 0.804 f 0.019 k=5 0.162 f 0.009‘ 0.763 f 0.010‘ 1.335 f O.01Zc 1.912 f 0.014‘ 2.254 f 0.014‘ 0.1 0.16 1 f 0.028 0.717 f 0.032 1.261 f 0.038 1.833 f 0.043 2.214 f 0.043 1 .o 0.095 f 0.028 0.412 f 0.029 0.773 f 0.032 1.301 f 0.038 1.894 f 0.042 5.0 0.041 f 0.028 0.149 f 0.028 0.281 f 0.028 0.533 f 0.030 1.092 f 0.035 From Table 1 for 0 + 0 with k = 2. ’ From Table 1 for 0 + 0 with k = 3. ‘ From Table 1 for 0 + 0 with k = 5.

TABLE 4

Selection limits (Rf standard error) numerically evaluated from (4) for unequal mutation rates with el = 0.80, cp = 0.15, and cs = 0.05 for three alleles [N= 20, a - Y(O,l)]

e s = 0.1 S = 0.5 s = 1.0 s = 2.0 S = 5.0

1 .o 0.036 f 0.016 0.174 f 0.016 0.362 f 0.016 0.731 f 0.015 1.268 f 0.015 the eventual contribution of mutation to response. tiple alleles as well). This condition requires 8 > l and This comparison may be best presented in terms of hence 8 >> S, which appears unlikely in real situations.] the ratio of the limit to initial response. Suppose that In the case that the initial population is a hybrid from we draw an initial population from an infinite equilib- two random inbred populations (with two alleles at a rium population with respect to mutation. The mean locus and each having frequency 0.5), the initial re- of this initial population is then expected to be zero sponse is sa: and the ratio with mutation becomes (since the mean of a is scaled to zero) and thegenetic 8N[1 - (1 + c2)/k]/(B+ l), which is about SN/(B + 1) variance is approximately 2a:[ 1 - (1 + c2)/k]. The as k + 00. Since the ratio without mutation is about initial responsefrom this population is thusabout 2N for weak selection, the eventual contribution of 2sa3 1 - (1 + c2)/k] and the ratio of the limit with mutationto the response to selection is therefore mutationto the initial response is 4N/(8 + 1). If, about 2N to 6N of the initial response for small 8, however, the initial population is from a finiteequilib- depending oninitial populations, and could be higher rium population with effective size N’, and has genetic than that if 8’ is also small. variance 2a31 - (1 + c2)/k]8’/(8’ + I), theratio As S increases the relative contribution of mutation becomes 4N(8’ + 1)/8’(8 + 1) where 8’ = 4N’p. This to response increases as well. To show this, let us ratio decreases as 8 and 8’ increase and is independent consider first the case of an initial hybrid population of the number of alleles. When 8’8 < 1, the ratio is with the selection parameter S (= 4Nia,/ap) = 5. By larger than 4N. [It is important to point out that, if using KIMURA’S(1 957) formula of fixation probability the parameters 8’ and 8 are such that (8’ + 1)/8‘(8 + for two alleles in conjunction with normally distrib- I) = 94, mutation will not increase selection response uted allelic effects, we can show that theselection limit since the ratiowithout mutation is about 2N for weak without mutationis about 0.8N of the initial response. selection (ROBERTSON1960) (2N ratio appliesfor mul- Thus by using (9) for 8 + 0 and k + CQ, the maximal M utation Effects on Selection on EffectsMutation Limits 983 mutation effect is calculated to be about 7.2N of the CHAKRABORTY,R., and M. NEI, 1982 Genetic differentiation of initial response (in this case the selection limit with quantitative characters between populations or species. I. Mu- tation and random genetic drift. Genet. Res. 39 303-3 14. mutation is 2aaf/suf = 8N of the initial response). CLAYTON,G. A., and A. ROBERTSON,1955 Mutation and quanti- If the initial population is from a finite equilibrium tative variation. Am. Nat. 89 151-158. population with mutation parameter 8’, the number CLAYTON,G. A., and A. ROBERTSON,1964 The effects of X-rays of alleles in a sample is likely to be more than two if on quantitative characters. Genet. Res. 5: 410-422. COCKERHAM,C. C., 1984 Drift and mutation with a finite number 8’ 2 0.5 (EWENS1972). To determine the selection of allelic states. Proc. Natl. Acad. Sci. USA 81: 530-534. response without mutationwith multiple initial alleles, COCKERHAM,C. C., and H. TACHIDA,1987 Evolution and main- we have carried out simulation experiments by using tenance of quantitative genetic variation by mutations. Proc. KIMURAand TAKAHATA’S(1983) improved sampling Natl. Acad. Sci. USA 84 6205-6209. technique (the detailed results will be presented else- EWENS,W. J., 1972 The sampling theory ofselectively neutral alleles. Theor. Popul. Biol. 3: 87-1 12. where). We used EWENS’ (1972) sampling probability EWENS,W. J., 1979 MathematicalPopulation Genetics. Springer- to determine theinitial frequencies for fixed numbers Verlag, New York. of initial alleles (2, 5, 10 and 20, which covers the HILL, W. G., 1982a Rates of change in quantitative traits from value of 8’ from about 0.2 to 5) and a - M(O,l),and fixation of new mutations. Proc. Natl. Acad. Sci. USA 79 142- found that the selection limits without mutation are 145. about N times the initial responses for S = 5 for all HILL,W. G., 1982b Predictions of response to artificial selection from new mutations. Genet. Res. 40 255-278. the initial allele numbers. Thus when S = 5, by using HILL,W. G., and J. RASBASH,1986 Models of long-term artificial (9) the eventual contribution of mutation to response selection in finite population with recurrent mutation. Genet. is Res. 48: 125-131. HOLLINGDALE,B., and J. S. F. BARKER,1971 Selection for in- 8 Nsu: creased abdominal bristle number in Drosophilamelanogaster 2sU,28’/(8’+ 1) - .I with concurrent irradiation. I. Populations derived from an inbred line. Theor. Appl. Genet. 41: 208-215. times the initial response, which is larger than JOHNSON, H. L., and S. KOTZ,1972 Distribution in Statistics: Con- tinuousMultivariate Distributions. John Wiley & Sons, New York. KIMURA,M., 1956 Rules for testing stability of a selective polymorphism. Proc. Natl. Acad. Sci. USA 42: 336-340. when S << 1. So it seems that for random neutral KIMURA,M., 1957 Some problems of stochastic processes in genetics. Ann. Math. Stat. 28 882-901. initial populations the maximal effect of mutation is KIMURA, M., 1965 A stochastic model concerning the mainte- about 2.8N to 23N of the initial response for 8’ nance of genetic variability in quantitative characters. Proc. ranging from 5 to 0.2. Natl. Acad. Sci. USA 54 731-736. From these examples we have seen the importance KIMURA,M., 1981 Possibility of extensive neutral evolution under of parameters 8’ and 8 on the effect of mutation on stabilizing selection with specialreference to non-random usage of synonymous codons. Proc. Natl. Acad. Sci. USA 78: 5773- the selection limit. The parameter 8’ does not influ- 5777. ence the selection limit, but does influence the initial KIMURA,M., and N. TAKAHATA,1983 Selective constraint in response. As 8’ decreases, initial response becomes protein polymorphism: Study of the effectively neutral muta- smaller and the ratioof response due to mutation over tion model by using an improved pseudosampling method. initial response will become larger. The parameter 8, Proc. Natl. Acad. Sci. USA 80 1048-1052. LANDE,R., 1975 The maintenance of genetic variability by mu- however, influences the limit, not the initial response. tation in a polygenic character with linked loci. Genet. Res. 26 As 8 increases, the limit becomes smaller. When 8 >> 221-235. S, the selection limit with mutation can even be re- LANDE,R., 1976 Natural selection and random genetic drift in duced to that without mutation. phenotypic evolution. Evolution 30: 314-334. LANDE,R., 1977 Statistical tests for natural selection on quanti- We thank M. TURELLI,W. G. HILL,B. WALSH,P. D. KEIGHTLEY, tative characters. Evolution 31: 442-444. C. H. LANGLEY andM. ~IZUKAfor comments on the earlier version LATTER,B. D. H., 1960 Natural selection foran intermediate of this paper. This is paper No. 11802 of the Journal Series of the optimum. Aust. J. Biol. Sci. 13: 30-35. North Carolina Agricultural Research Service, Raleigh, North Car- LI, W.-H., 1977 Maintenance of genetic variability under muta- olina 27695-7643. This investigation was supported in part by tion and selection pressures in a finite population. Proc. Natl. Research Grant GM 1 1546 from the National Institute of General Acad. Sci. USA 74 2509-2513. Medical Sciences. LYNCH,M., 1988 The rate of polygenic mutation.Genet. Res. 51: 137-148. LITERATURECITED LYNCH,M., and W. G. HILL, 1986 Phenotypic evolution by neutral mutation. Evolution 40 915-935. BARTON,N. H., 1986 The maintenance of polygenic variation MUKAI,T., 1979 Polygenic mutation, pp. 177-195 In Quantitative through a balance between mutation and stabilizing selection. Genetic Variation, edited by J. N. THOMPSON,JR., and J. M. Genet. Res. 47: 209-2 16. THODAY.Academic Press, New York. BULMER,M. G., 1972 The genetic variability of polygenic char- ROBERTSON,A., 1960 A theory oflimits in artificial selection. acters under optimizing selection, mutation, and drift. Genet. Proc. R. SOC.Lond. Ser. B 153: 234-249. Res. 19 17-25. SLATKIN,M., 1987 Heritable variation and heterozygosity under 984 Z.-B. Zeng, H. Tachidaand C. C. Cockerham

abalance between mutationsand stabilizingselection. Genet. WATTERSON,G. A,, 1977 Heterosis or neutrality? Genetics 85: 5 3-62. 789-8 14. 789-8 Res. 50: 53-62. TURELLI,M., 1984 Heritablegenetic variation via mutation-selec- WRIGHT,S., 1949 Adaptationand selection,pp. 365-389 In tion balance: Lerch’s zeta meets the abdominal bristle. Theor. Genetics, Paleontology and Evolution, edited by G. L. JEPSON,G. Popul. Biol. 25: 138-193. G. SIMPSONand E. MAYR.Princeton University Press, Prince- TURELLI,M., J. H. GILLEPSIEand R. LANDE,1988 Rate tests for ton, N.J. selection on quantitative characters during macroevolution and m icroevolution. Evolutionmicroevolution. 42: 1085-1089. Communicatingeditor: M. TURELLI