Copyright 0 1987 by the Genetics Society of America Directional Selection and Variation in Finite Populations Peter D. Keightley and ‘William G. Hill Institute of Animal Genetics, University of Edinburgh, West Mains Road, Edinburgh EH9 3JN, Scotland Manuscript received April 12, 1987 Revised copy accepted July 16, 1987 ABSTRACT Predictions are made of the equilibrium genetic variances and responses in a metric trait under the joint effects of directional selection, mutation and linkage in a finite population. The “infinitesimal model“ is analyzed as the limiting case of many mutants of very small effect, otherwise Monte Carlo simulation is used. If the effects of mutant genes on the trait are symmetrically distributed and they are unlinked, the variance of mutant effects is not an important parameter. If the distribution is skewed, unless effects or the population size is small, the proportion of mutants that have increasing effect is the critical parameter. With linkage the distribution of genotypic values in the population becomes skewed downward and the equilibrium genetic variance and response are smaller as disequilibrium becomes important. Linkage effects are greater when the mutational variance is contributed by many genes of small effect than few of large effect, and are greater when the majority of mutants increase rather than decrease the trait because genes that are of large effect or are deleterious do not segregate for long. The most likely conditions for ‘‘Muller’s ratchet” are investi- gated. N recent years there has been much interest in the is attained more quickly in the presence of selection I production and maintenance of variation in pop- than in its absence, and is highly dependent on popu- ulations by mutation, stimulated by the presence of lation size. Thus in the early generations of a selection abundant variation in natural and artificial popula- experiment or breeding program the response from tions at the protein and DNA levels. Also, the genome variation generated by new mutations is expected to is now seen as a fluid entity with transposition a be small. In later generations, however, the contri- particularly potent force in generating molecular var- bution to the total variation present and hence to the iability. Variation at the phenotypic level must also response can be very important, especially in large originate from mutation, but the rate at which such populations. The results of long-term selection exper- variation is generated has been thought to be slow. iments can be interpreted in light of these analyses. This belief was derived mainly from observations of The continued response after 120 generations of di- experimental populations of Drosophila. For example, rectional selection for increased pupal weight in Tri- the gain from new mutations in bristle score variation bolium (ENFIELD1980), after at least 75 generations is of the order of one thousandth of the environmental of selection for increased bristle score of Drosophila variation per generation (discussed by LANDE1976; (Yo0 1980), and after 76 generations for increased HILL 1982b), and mutagenesis experiments have oil content in maize (DUDLEY1977) were likely to failed to produce large amounts of new variation in have been strongly influenced by variation arising such quantitative traits (CLAYTONand ROBERTSON while the experiment was proceeding. 1964; KITACAWA 1967; HOLLINCDALEand BARKER In a previous paper (KEIGHTLEYand HILL 1983) 1971). Monte Carlo simulation was used to investigate the Despite the apparent slowness of accumulation of effect of linkage on asymptotic selection responses in new mutational variance, theoretical analyses of the small populations with new mutations and it was con- interaction of mutation and natural selection in the cluded that the asymptotic response rate is little af- absence of drift have shown that mutation may be a fected by linkage, especially for species with many powerful force in maintaining variation in natural chromosomes. Moreover, the variance of effects of populations (LANDE1976), although the extent pre- mutants on the trait was not an important parameter dicted depends on assumptions in the model (TURELLI of the model for asymptotic selection responses were 1984). Theoretical studies in finite populations have little affected by whether the new mutational variance concentrated on the combined effect of mutation and arose from a few genes of large effect or many genes directional selection in influencing quantitative vari- of small effect. Mutant effects on the trait were, ability and selection response rates (HILL 1982a,b). however, assumed to be symmetrically distributed The equilibrium variance of a quantitative character about zero. Genetics 117: 573-582 (November, 1987) 574 P. D. Keightley and W. G. Hill In this paper we attempt to develop a theoretical ation, t, the above three quantities are related by framework to predict the amount of quantitative var- vA,t = vg,t iation maintained and selection responses in finite + cot. (2) populations from the simultaneous segregation of There is also a covariance component due to devia- newly arising linked mutations. The analysis is based tions from Hardy-Weinberg equilibrium (BULMER on the ‘infinitesimal model’ first used by FISHER 1976). We ignore this in the model since it is transient (19 18), which provides a reference point to allow us and disappears after one generation irrespective of to better understand the results of the more complex linkage . Monte Carlo simulation. Change of variance As an extension of the previous work, we shall also investigate cases where the effects of new mutations In an isogenic population, all three quantities V,, VA come from a skewed distribution, i.e., mutations of and ZD are zero. With constant forces of mutation, increasing or decreasing effect on the character are selection and drift, they approach equilibria when the not equally likely. While it may be reasonable to rate of loss of variation due to selection and drift is assume that mutant effects are symmetrically distrib- balanced by the rate of gain from new mutations. In uted for bristle number, it is likely that mutations a finite population the variances will drift stochasti- affecting characters close to fitness are mainly detri- cally about the equilibrium. A prediction of the infin- mental (MUKAIet al. 1972). We also discuss the impli- itesimal model is that the genic variance (V,) is unaf- cation of the behavior of the model in theories of the fected by selection (see CROWand KIMURA1970, pp. evolutionary advantages of recombination, and in par- 236-239; BULMER1980, Ch.9) and therefore in an ticular, investigate the conditions necessary for the infinite population with mutation V, will never reach operation of “Muller’s ratchet” (MULLER 1964; FEL- an equilibrium. Designating VM the expected incre- SENSTEIN 1974), where the population mean can de- ment in variance in the population per generation cline due to the fixation of recurrent mutations of from new mutations, the expected change in genic negative effect. variance in one generation is given by V,,t+I = V,,t(l - 1/2N) + VM, (3) ANALYSIS OF THE INFINITESIMAL MODEL where N is the effective population number. In a finite Definitions population, the expected equilibrium value of V, is 2” (cf CLAYTONand ROBERTSON1955). The quantitative character is assumed to be deter- The equilibrium value of the additive genetic vari- mined by the sum of an infinite number of genes of ance is affected by selection, mutation, drift and re- infinitely small effects. Dominance and epistasis are combination, and is less simply derived than that of assumed to be absent and expression are given for the the genic variance. As a starting point, we consider case of two alleles per locus. At the ith locus, a, is the effects of selection in an infinite population with defined as the difference between homozygotes and free recombination in the absence of mutation. qz is the frequency of the higher valued allele. Selection: The effect of selection on genetic varia- We utilize the following symbols for three funda- bility in the infinitesimal model has been discussed by mental quantities which are essentially the same as BULMER(1971, 1980, pp. 153-154), and by FAL- defined by BULMER(1 976): CONER (1981, pp. 179-189). After one generation of V, = Za?q,(l - qJ/2 is the genic variance in the selection of parents and breeding of progeny, the total population. It is the genetic variance which would be genetic variance in the population can be divided into present if the frequencies of each individual gene were fractions between and within full-sib families. With measured and it were assumed that both correlations random mating, the between family component is one- between loci due to linkage and deviations from half of the genetic variance among parents. Selection Hardy-Weinberg equilibrium were zero. by truncation reduces the variance in the parents by VAis the additive genetic variance in the population. a factor of 1 - h2k, where h2 is the heritability (h2 = It is the variance of breeding values among individ- VA/VP,the squared correlation between phenotype uals. and genotype; with selection on an arbitrary index h2 is replaced by the squared accuracy of the index), and k is a constant factor describing the strength of selec- tion. For truncation selection of a normally distrib- is the disequilibrium covariance in the population, uted population, K = i(i - x), where i is the intensity where D, = J3 - q,q,, and J3 is the frequency of the of selection (standardized selection differential) and x corresponding gamete. It is the sum of the covariances is the standardized deviation of the truncation point. between the values of pairs of loci in the population, Thus after one generation, the between family com- and can be either negative or positive. At any gener- ponent is given by (1 - h2k)VA/2. Variation in Finite Populations 575 Selection leads to a reduction in the genetic variance ponent is obtained by combining equations (2)and (7) between family means, which appears as a negative to give disequilibrium covariance component within families.