Response to Selection in Finite Locus Models with Nonadditive Effects

Home , Additive genetic effects, Infinitesimal model

Journal of Heredity, 2017, 318–327 doi:10.1093/jhered/esw123 Original Article Advance Access publication January 12, 2017

Original Article Response to Selection in Finite Locus Models with Nonadditive Effects

Hadi Esfandyari, Mark Henryon, Peer Berg, Jørn Rind Thomasen, Downloaded from https://academic.oup.com/jhered/article/108/3/318/2895120 by guest on 23 September 2021 Piter Bijma, and Anders Christian Sørensen

From the Center for Quantitative Genetics and Genomics, Department of Molecular Biology and Genetics, Aarhus University DK-8830 Tjele, Denmark (Esfandyari, Berg, Thomasen, and Sørensen); Seges, Danish Pig Research Centre, Copenhagen, Denmark (Henryon); School of Animal Biology, University of Western Australia, Crawley, Australia (Henryon); Nordic Genetic Resource Center, Ås, Norway (Berg); VikingGenetics, Assentoft, Denmark (Thomasen); and Animal Breeding and Genomics Centre, Wageningen University, Wageningen, The Netherlands (Bijma).

Address correspondence to H. Esfandyari at the address above, or e-mail: hadi.esfandyari@ mbg.au.dk.

Received May 18, 2016; First decision June 7, 2016; Accepted January 2, 2017.

Corresponding Editor: C Scott Baker

Abstract Under the finite-locus model in the absence of mutation, the additive genetic variation is expected to decrease when directional selection is acting on a population, according to quantitative-genetic theory. However, some theoretical studies of selection suggest that the level of additive variance can be sustained or even increased when nonadditive genetic effects are present. We tested the hypothesis that finite-locus models with both additive and nonadditive genetic effects maintain more additive genetic variance (VA) and realize larger medium- to long-term genetic gains than models with only additive effects when the trait under selection is subject to truncation selection. Four genetic models that included additive, dominance, and additive-by-additive epistatic effects were simulated. The simulated genome for individuals consisted of 25 chromosomes, each with a length of 1 M. One hundred bi-allelic QTL, 4 on each chromosome, were considered. In each generation, 100 sires and 100 dams were mated, producing 5 progeny per mating. The population was selected for a single trait (h2 = 0.1) for 100 discrete generations with selection on phenotype or BLUP-EBV. VA decreased with directional truncation selection even in presence of nonadditive genetic effects. Nonadditive effects influenced long-term response to selection and among genetic models additive gene action had highest response to selection. In addition, in all genetic models, BLUP-EBV resulted in a greater fixation of favorable and unfavorable alleles and higher response than phenotypic selection. In conclusion, for the schemes we simulated, the presence of nonadditive genetic effects had little effect in changes of additive variance and VA decreased by directional selection.

Subject area: Quantitative genetics and Mendelian inheritance Keywords: finite locus model, nonadditive effects, selection

Dominance and epistatic effects have been detected in many experi- (VA) in populations (Goodnight 1987; Cheverud and Routman mental populations (Carlborg and Haley 2004) and these allelic 1995; Hansen and Wagner 2001; Caballero and Toro 2002; Barton interactions can influence the amount of additive genetic variance and Turelli 2004). It has been argued that epistatic variance may be

“converted” into additive variance by genetic drift when a population population size (Robertson 1960). In the absence of new mutations, passes through a population bottleneck (Goodnight 1995; Cheverud the total response to selection is limited by the initial standing vari- and Routman 1996; Cheverud et al. 1999; López-Fanjul et al. 2002; ation. Selection acts on the variation present in the population, and Barton and Turelli 2004). However, this argument is not restricted will typically act to reduce it; the extent to which it does so depends to genetic drift. Changes in VA occur with variations in the genetic on the allele frequencies, and on the relation between genotype and background, and any process that changes allele frequencies, includ- phenotype (i.e., the “genetic architecture”). Interactions between ing selection, can change VA (Hansen and Wagner 2001). Gene inter- alleles (epistasis) are a key component of this genetic architecture, actions may also affect response to selection through a build-up of and their effect on the response to selection has been controversial. linkage disequilibrium associated with favorable gene combinations, On the one hand, artificially selected populations are usually well since parents transmit not only half of the additive effects to offspring, approximated by the infinitesimal model, which, in its simplest form, but also a quarter of the pairwise epistatic effects (A × A) and smaller assumes infinitely many loci of small and additive effect (Hill et al. fractions of higher-order interactions (Lynch and Walsh 1998). This 2008). This suggests that gene interactions do not play an important suggests that some of the linkage disequilibrium built by epistatic role. On the other hand, biology can hardly be additive, and genomes selection can be converted into response to selection (Griffing 1960). are finite. If the trait depended on a small number of additive alleles, Downloaded from https://academic.oup.com/jhered/article/108/3/318/2895120 by guest on 23 September 2021 The model most commonly used for genetic evaluation is the these would quickly fix contradicting the robust observation of sus- infinitesimal model. It assumes large numbers of genes affecting traits tained response to selection. One possibility is that epistasis sustains with each gene having a small additive effect (Fisher 1918). In this additive genetic variance for longer: Alleles that were initially del- model, selection does not change the allele frequency significantly at eterious or near-neutral may acquire favorable effects as the genetic any individual locus, nor does it change the genetic variance except background changes, “converting” epistatic variance into additive, for nonpermanent changes caused by gametic phase disequilibrium and so prolonging the response to selection. The question of whether (the Bulmer effect; Bulmer 1971). The assumptions of the infini- selection can generate additive genetic variance is an important one tesimal model are incorrect. In practice, there are individual genes, for a number of reasons, including the more nuanced understanding sometimes with large effects, and many genes showing dominance we are developing of the relationship between molecular variation and epistasis (Mackay, 2001a, 2001b). An alternative to the infinites- (“QTL”) and phenotypic variation, and the results of long-term selec- imal model is a finite-locus model, which can accommodate nonad- tion experiments that can show an approximately linear response to ditive inheritance. Under the finite-locus model, the additive genetic selection for a hundred generations or more (Goodnight 2015). variation is expected to decrease when directional selection is acting Previous studies for investigating effect of selection on additive on a population, according to quantitative-genetic theory (Crow and genetic variance in presence of nonadditive effects have focused on Kimura 1970; Falconer and Mackay 1996). However, some theoreti- phenotypic selection (Fuerst et al. 1997; Carter et al. 2005; Hallander cal studies of selection suggest that the level of additive variance can and Waldmann 2007). An alternative to selection based only on the be sustained or even increased when nonadditive genetic effects are phenotypic record of the individual is selection based on best linear present, in a manner similar to the action of genetic drift (Carter unbiased predictor (BLUP) of breeding value (Henderson 1975), et al. 2005; Fuerst et al. 1997). Experimental evidence for this phe- which uses records on all relatives, in addition to the individual’s own nomenon was found by Martinez et al. (2000), when they selected record, in genetic evaluation. The BLUP theory was first proposed by mice for body fat, and by Sorensen and Hill (1982), who selected Henderson in 1949 to describe the environmental and genetic trend D. melanogaster for abdominal bristle number. Furthermore, estimates in cattle and was subsequently developed (Henderson 1975). Carlborg et al. (2006) showed that epistatic interactions between 4 The main advantage of the BLUP methodology is the high level of loci mediated a considerably higher response to selection for growth accuracy of the breeding (or genetic evaluations) and has the property in chicken than predicted by a single-locus model. In a simulation to account for selection of parents in a breeding population. Hence, it study, Hallander and Waldmann (2007) investigated changes inVA in fairly accounts for the fact that some animals are from better parents the presence of nonadditive effects in a trait subjected to directional than other animals. Many simulation studies have compared selection selection. They showed that by including dominance and epistatic responses based on BLUP with those based on either phenotype or effects, VA was increased during the initial generations of selection. family index. In 2 simulation studies of a pig population, phenotypic Fuerst et al. (1997) showed similar results by using a 2-locus genetic selection led to response rates of 64% and 91% of those for BLUP for model to simulate a trait with different levels of additive and non- heritabilities of 0.1 and 0.6, respectively (Belonsky and Kennedy 1988). additive genetic effects. In these simulation studies VA increased by However, for a trait under selection, there is no theory to quan- including nonadditive effects, but it seems considering 2–4 loci with tify the differences between phenotypic selection or BLUP-EBV equal additive and dominance effects across all loci with initial fre- selection when the trait is controlled both by additive and nonad- quency of 0.5 is not a realistic model of the underlying genes. In fact, ditive genetic effects. This work explores the effect of presence of finite-locus models used in these studies to test quantitative theory nonadditive effects in genetic models and assesses their importance are too restrictive. Indeed, it is possible to fit less restrictive models in medium- to long-term selection experiments. For this objective, with many genes each having a unique effect, thus allowing a range we examine how the genetic variance and genetic gain are affected from genes of large to zero effect. These genes could display nonad- by the presence of nonadditive genetic effects, using BLUP-EBV and ditive effects such as dominance or epistasis. In theory, such a model phenotypes as selection criteria. seems to agree more closely with what we know about the genetics of quantitative traits than the simple models (Goddard 2001). Methods In animal breeding, response to short-term selection of a few generations depends on the intensity and accuracy of selection. Response Procedure to long-term selection depends on how much genetic variability, in We used stochastic simulation to estimate genetic gain and moni- particular, additive variance is maintained and how much inbreeding tor changes in genetic-variance components generated by 4 genetic depression occurs over generations, and therefore on effects effective models, with 2 different selection criteria and 2 distributions of QTL 320 Journal of Heredity, 2017, Vol. 108, No. 3

effects. The models were applied to a population undergoing direc- why selection response does not decline in the first few generations, tional truncation selection for a single trait over 100 discrete genera- because selection increases the frequency of rare favorable alleles tions. During each generation, the genetic gain and changes in additive, and, hence, increases the genetic variance due to these loci, which dominance, and epistasis variances were calculated. The simulations compensates for the loss of variance caused by selection for common were carried out using a modified version of ADAM Pedersen( et al. favorable alleles (Goddard 2001).

2009). For each scenario, 100 replicates were performed. Two distributions were fitted for QTL effects a( i); either a gamma distribution (0.4, 1.66) or a mixture of a double exponential distri- Genetic Models bution and a normal distribution, that is, The 4 genetic models, 2 selection criteria, and 2 distributions of QTL a ~.0950.,Lu2 00..50Nu, 5 2 effects are presented in Table 1. The first genetic model (A) assumed i ()+ ()() that the trait is controlled by additive-gene actions. In the second model (A/D), the trait was controlled by additive and dominance (Figure 1). With appropriate u > 0. These distributions resulted effects. In the third model (A/AA), additive and additive × addi- in many QTL with small effects and few QTL with large effect tive epistatic effects were included. In the final model, a full-genetic Downloaded from https://academic.oup.com/jhered/article/108/3/318/2895120 by guest on 23 September 2021 (Bennewitz and Meuwissen 2010). Dominance effects (di) were gen- model (A/D/AA) consisted of additive, dominance, and additive × erated between alleles at the same locus and interactions between additive epistatic effects. loci (aaij) only occurred for each pair of neighboring loci, and a locus

had only interaction with one other locus. Dominance degrees (hi) Selection Criteria were normally distributed with mean µh = 0.5, variance Vh = 1, and Truncation selection was applied in each generation and the criterion they were independent of the additive effects. Dominance effects were for truncation selection was either the phenotypic observation of the then calculated as dhii= ai so they became dependent to the addi- individual or estimated breeding values (EBV) obtained from standard tive effects (Wellmann and Bennewitz 2011). First-order (additive-

BLUP evaluations. The BLUP-EBVs were obtained by Henderson’s by-additive) epistatic degrees (kij) also followed a normal distribution

(1975) mixed linear model. Two pieces of information were used: (0, 1), and the epistatic effects were calculated as aakij =⋅ij aaij, phenotypic records and pedigree data. The following model was used where aaij is the epistatic effect of the 2 adjacent loci, ki is the epistatic to estimate breeding values: y = µ + Zu + e, where y is a vector of phe- degree, ai and aj are the additive effects at the first and second locus. notypic values, µ is the overall mean Z is known design matrices relat- The total genotypic value of an individual was obtained by ing observations to the random effects u, u is the vector of breeding summing the genotypic contribution of each locus pair (Table 2). values, and e is the vector of random residual effects. The numerator relationship matrix (A) was used in the following mixed model equa- tions to derive BLUP of random additive effects (breeding values):  σ 2  ZZ AZ−1 e aˆ []y  ′ + 2 []= ′  σ a 

2 2 where σ e is the residual variance and σ a is the additive genetic variance.

Simulated Genome and Distributions of QTL Effects The simulated genome consisted of 25 chromosomes. One hundred QTL, 4 on each chromosome, were considered. All chromosomes had a length of 1 M and the QTL were assumed to be positioned randomly on each chromosome following a uniform distribution. All QTL were bi-allelic and initial frequencies of the alleles followed a U-shaped distribution as suggested by Crow and Kimura (1970). The U-shaped distribution of gene frequencies explains

Table 1. Values of the initial ratio of nonadditive variances on additive variance, QTL effects distribution and selection criteria in simulated scenarios Figure 1. Distribution of QTL effects. Tested parameters Values

Distribution of QTL effects Mixture, Table 2. Genetic effects of genotypes for a pair of loci based on Gamma Fuerst et al. (1997) Selection criteria P, BLUP Genotype at locus 1 Genotype at locus 2 Genetic model VD/VA VAA/VA A 0 0 A/D 1/2 0 BB Bb bb A/AA 0 1/4 AA a + a + aa a + d a − a − aa A/D/AA 1/2 1/4 1 2 12 1 2 1 2 12 Aa d1 + a2 d1 + d2 d1 − a2

aa −a1 + a2 − aa12 −a1 + d2 −a1 − a2 + aa12 P stands for phenotypic selection. Journal of Heredity, 2017, Vol. 108, No. 3 321

The genetic variance components depend on gene frequencies and normal distribution for each individual with mean zero and variance values of different genetic effects. Following Fuerst et al. (1997), they VE = Vp (VA + VD + VAA). were computed as:

np 2 VA =+∑{2pq11aq11()− pd11+−()pq22aa12  Results l (1) 2 2pq aqp dpqaa } Additive Variance ++22 22( − 2 ) 21+−()112  l In the following, results for the Mixture distribution of QTL effects np are not presented as we found similar and consistent results for the 2 2 2 2 2 2 VD =+∑{}44pq1 1 dp1 2 qd2 2 l (2) 2 distributions (gamma vs. mixture) fitted for QTL effects. In order l to compare trends of VA across scenarios, additive genetic variance np 2 in each scenario was scaled by dividing all values to initial addi- VAA = ∑{}4pq11pq22aa12 l (3) l tive variance of each scenario. Figure 2 shows changes of VA over generations in 4 genetic models. There was a systematic pattern in the loss of V when comparing the 4 genetic models, and differences Downloaded from https://academic.oup.com/jhered/article/108/3/318/2895120 by guest on 23 September 2021 where VA is the additive variance (variance of breeding values), np A between genetic models were relatively small. In the A/AA model, the is the total number of pairs of loci, VD is the dominance variance loss of V was faster than in the other genetic models. Initially, the (variance of dominance deviations), VAA is the additive-by-additive A decline in V was smallest in the A-model. In the long-term, however, variance, a1 and a2 are the additive effect at loci 1 and 2, d1 and d2 A the amount of retained V was highest in full genetic model (A/D/ are the dominance effects, and aa12 is the additive-by-additive effect A at the pair (1 and 2). The gene frequencies of alleles A, B, a, and b AA). Two distinct selection criteria showed differences in reduction of V over time, as the BLUP-EBV selection criteria accelerated the are p1, p2, q1, and q2. A

reduction in VA. Population Structure One hundred sires and 100 dams were in the base population (gen- Dominance Variance eration 0) of each scenario and were mated randomly to produce the In the 2 genetic models, A/D/AA and A/D, where dominance effects first generation of offspring. Offspring produced by the base popula- were included, VD increased or was constant in the initial generation were selected in generation 1 and the first generation of offspring tions (Figure 3). Afterwards VD decreased in the A/D model. Changes from selected parents was produced in generation 2. Offspring pro- in VD over time differed for the 2 selection criteria. When selection duced in generation 100 were the result of 99 generations of selec- was based on BLUP-EBV, the reduction of VD over time was faster tion. In generation 1 through 100, 200 animals (the top 100 males than with phenotypic selection. A striking result was that phenotypic and top 100 females) were selected from the 500 available candi- selection hardly affected VD in the A/D/AA model over the entire dates in each generation, based on their phenotype or BLUP-EBVs. period of 100 generations. Thus, the selected proportions were 40% (100 out of 250) in males and 40% in females (100 out of 250). Each sire was mated with 1 Additive-by-Additive Genetic Variance dam, and 5 full-sib offspring were produced per mating, resulting in The epistatic variance decreased in a similar way over the 100 500 offspring in each generation. generations of selection in both genetic models and both selection criteria (Figure 4). The decrease was fastest for selection on BLUP- Trait EBV, where VAA approached its lowest level (~0) at generation 60.

The trait under selection had a narrow-sense heritability of 0.10 in For phenotypic selection, longer time was needed for VAA to vanish the base population. The environmental values were sampled from a completely.

Figure 2. Changes in additive genetic variance (VA) in 4 genetic models separated by selection criteria. The plotted additive variances for each scenario are means from 100 replicates, standardized by the initial additive variance of each scenario. (a) Selection method: phenotypic and (b) selection method: BLUP-EBV. 322 Journal of Heredity, 2017, Vol. 108, No. 3 Downloaded from https://academic.oup.com/jhered/article/108/3/318/2895120 by guest on 23 September 2021

Figure 3. Changes in dominance genetic variance (VD) in 2 genetic models (A/D/AA and A/D) separated by selection criteria. The plotted dominance variances for each scenario are means from 100 replicates, standardized by the initial dominance variance of each scenario. (a) Selection method: phenotypic and (b) selection method: BLUP-EBV.

Figure 4. Changes in additive-by-additive genetic variance (VAA) in 2 genetic models (A/D/AA and A/AA) separated by selection criteria. The plotted additive by additive variances for each scenario are means from 100 replicates, standardized by the initial additive by additive variance of each scenario. (a) Selection method: phenotypic and (b) selection method: BLUP-EBV.

Response to Selection percentage of favorable alleles that became fixed was higher than per- The mean observed genotypic value of individuals in each genera- centage of favorable alleles that were lost. Of the total fraction of fixed tion, expressed in initial additive genetic standard deviations and as alleles, however, the additive model had a greater proportion of loci a deviation from the initial mean, is plotted in Figure 5. As selec- fixed for the favorable allele. BLUP-EBV generated higher levels of tion proceeds, the difference between genetic models became larger. fixation and loss of favorable alleles than phenotypic selection, which In generation 100, the additive model had the highest cumulative agrees with the earlier plateau of response seen with BLUP selection in response to selection, and the full genetic model (A/D/AA) had the Figure 5. In addition, the ratio of favorable fixed over total allele fixa- lowest cumulative response in long-term. When selection was based tion was higher in BLUP-EBV than phenotypic selection. on BLUP-EBV, for all genetic models response plateaued earlier than for phenotypic selection. Discussion Fixation and Loss of Favorable QTL Our findings did not support the hypothesis that finite-locus models

The additive genetic model (A) had the highest percentage of fixation with both additive and nonadditive genetic effects maintain more VA of favorable alleles at generation 100 (Figure 6). This result agrees and realize larger medium- to long-term genetic gains than models with the highest response found for the additive model in Figure 5. with only additive effects when the trait under selection is subject to The percentage of fixed favorable alleles decreased when more non- truncation selection. We used 4 genetic models to simulate a popula- additive effects were added to the model. In all genetic models, the tion undergoing directional truncation selection. In all 4 models, VA Journal of Heredity, 2017, Vol. 108, No. 3 323 Downloaded from https://academic.oup.com/jhered/article/108/3/318/2895120 by guest on 23 September 2021

Figure 5. Response to selection over 100 generations of selection separated by selection criteria. The plotted mean genotypic values for each scenario are means from 100 replicates, standardized by initial additive genetic standard deviations of each scenario. (a) Selection method: phenotypic and (b) selection method: BLUP-EBV.

1990; Fuerst et al. 1997). Directional selection changes the mean of a trait and it can also change the variance. First, directional selection

decreases VA due to the generation of negative gametic phase disequilibrium (Bulmer 1971). In the intermediate term, regardless of selection criteria and QTL distribution, genetic models with epistatic

terms (A/AA and A/D/AA), showed faster reduction in VA compared to the purely additive model (Figure 2). One explanation for this could be that the double homozygote genotype is more favored by

selection and that reduces VA by inducing negative linkage disequilibrium among selected genes. If the selected genes are linked, the

decay of linkage disequilibrium is delayed, and the reduction of VA is enhanced (Nomura 2005). However, it has been shown in several

Figure 6. Percentage of allele fixation and lost at generation 100 in 4 genetic studies as well as experimental results that epistatic variance and to models separated by selection criteria. Fav. fix: percentage of QTL in which some extent dominance variance might convert to additive genetic frequency of favorable allele is 1 in generation 100, Fav. lost: percentage of variance (Fuerst et al. 1997; Hallander and Waldmann 2007). In QTL in which frequency of favorable allele is 0 in generation 100. addition, any changes in VA might depend on the ratio of VAA to VA.

If VAA is smaller than VA, as in our case, then epistatic values for a decreased by directional selection, also in the presence of nonaddi- pair of loci might be small and this will cause more decrease in VA tive effects, but the rate at which variation decreased varied among by selection in comparison of purely additive gene action (Mueller genetic models and selection criteria. and James 1983). In order to figure out the impact of initial ratio of The main reason that we did not found any enhancement of the VAA over VA on both response to selection and changes in VA we did additive genetic variance due to nonadditive effects may be explained extra analysis. We run Model A/AA with varying the ratio of VAA to as the following. First, the ratio of VAA/VA used in this study was 1/4, VA in the model. The VAA was considered to be half, equal and twice however, Goodnight (1987) showed that the additive genetic variance of the VA (Supplementary Figure 1). The results confirmed the impact was not expected to increase unless the VAA was at least 1/3 of the VA. of initial ratio on response to selection. Increasing the VAA/VA ratio, Also, epistatic effects in our simulation were nondirectional. It has been resulted in higher response to selection, both in short- and long-term, shown that pure nondirectional epistasis has no significant effects on however, VA decreased in the all ratios considered, nonetheless the the response and additive variance (Carter et al. 2005). Furthermore, reduction in VA was faster when VAA was twice the additive variance. in our simulation the allele frequencies were drawn from a U-shaped Second, a more important cause for changes in VA is due to frequency distribution. Epistatic variance will only show up as a meas- changes in allele frequency. If all alleles are at intermediate fre- urable variance component when the allele frequencies are near 0.5. In quencies, it can be expected that variance will decline monotoni- summary, we believe that discrepancy of our results compared to those cally with time (assuming additivity), whereas if some are at low of the previous models (Fuerst et al. 1997; Hallander and Waldmann frequency, an initial increase in variance might be observed. If 2007) investigating the effect of nonadditive effects on response to selec- the distribution of frequencies is U shaped, as in this study, then tion was due to less restrictive assumptions in our simulated models. the increase in variance due to alleles at low frequency might be expected to outweigh the decrease from those at high frequency Changes in Variance (Hill and Bünger 2010). However, analyses undertaken by Hill

In our finite locus model, VA decreased by selection, which is in and Rasbash (1986) for finite populations indicate that the pat- agreement with results from other studies (Villanueva and Kennedy tern of response and change in VA is somewhat robust to the gene 324 Journal of Heredity, 2017, Vol. 108, No. 3

frequency distribution and VA decreases eventually as favorable no overdominance or its percentage was small (10%), dominance alleles are moved to fixation. variance decreased over generations, while increasing amount of

Third, in finite populations,V A also declines due to drift and, in overdominance in the model resulted in sustained dominance vari- the absence of selection, this decline can be predicted as VAt = VA0 ation over generations. This result implies that occurrence of over- t (1 − 1/2Ne) when assuming additive-gene action (Robertson 1960), dominance can serve as an explanation for the maintenance of VD where VAt is additive variance at generation t, VA0 is initial addi- in our study. tive variance, and Ne is effective population size. We applied the Similar to VA, epistatic variance decreased by directional selec- approach developed by PerezEnciso (1995) to estimate Ne. By tion in this study. Contrary to VD that its changes over time was regressing ln()1 − Ft on generation tt, Ne is estimated from Ne = ½(1 − affected by the amount of overdominance, changes in VAA was not β e ), where β is the slope from the regression of ln()1 − Ft on generation affected neither by the type of epistasis effects (e.g., directional or number and Ft is the average inbreeding coefficients at generation t nondirectional epistasis) nor the ratio of VAA/VA (results not shown).

(PerezEnciso 1995). We plotted VA predicted by this formula versus In all the models including epistasis effects, we found a reduction in observed VA for the additive model (A) to see how they would dif- epistatic variance that is probably due to the fixation of the loci over fer (Figure 7). The large difference between predicted and observed the course of selection. Downloaded from https://academic.oup.com/jhered/article/108/3/318/2895120 by guest on 23 September 2021

VA in Figure 7 is due to the fixation of favorable alleles by selection, as the formula based on effective population size (or inbreeding) assumes changes the allele frequency due to drift only. The lines Response to Selection for predicted VA in Figure 7 show that, as expected, selection based Our results demonstrate that nonadditive effects may affect response on BLUP-EBV generates more fixation due to drift than phenotypic to selection in long-term. Comparing genetic models, A and A/AA selection and as pointed out earlier fraction of favorable fixed over models had higher response in long-term than the models having total fixation was higher in BLUP-EBV. a dominance component (Figure 5). The reason for the greater

In contrast to VA, which decreased by selection, VD was preserved response is probably due to the constellation of the genotypic values, especially by phenotypic selection over generations. This mainte- where one double homozygote pair of loci has higher value. Hansen nance of dominance variation can be explained by the occurrence of and Wagner (2001) argued that nonadditive effect such as direc- overdominance at some loci. In the A/D/AA genetic model, around tional epistasis will affect the response to selection due to systematic 25% and 20% of loci showed overdominance for the mixture and changes in the effects of alleles as their genetic background changes. gamma distribution, respectively. It has been shown that overdomi- On the other hand, if the epistatic interactions are random and non- nance results in stabilizing selection, maintaining heterozygosity in directional as in this simulation, these effects will tend to cancel out the population rather than driving one allele to fixation. In order to or add random noise. However, over many generations, the dynam- investigate the effect of amount of overdominance on the preser- ics of gene effect reinforcement and competition can become very vation of dominance variance over generations, we run model A/D complex, and may lead to substantial departures from simple addi- with varying amount of loci that showed over dominance. So, 4 tive response to selection (Carter et al. 2005). We run model A/D/AA cases of A/D model was considered in which 0 (overdominance was when epistatic effects in the model were allowed to be both direc- not allowed), 10%, 20%, and 30% of loci were allowed to have over tional and nondirectional (i.e., positive and negative). We found that dominance. All other parameters in the model such as ratio of domi- directional epistasis enhanced the response to selection in long-term nance variance to additive variance were kept constant and similar (Supplementary Figure 6). to the original model described in the Methods section. The results One deduction of these results could be that estimates of clas- showed clear effect of amount of overdominance on the changes sical epistatic variance components are of little value in predicting of dominance variance (Supplementary Figure 2). When there was response in short term, as these estimation do not distinguish between

Figure 7. Comparison of changes in observed VA in pure additive genetic model with VA based on inbreeding calculated as VAt = VA0 (1 − 1/2Ne). (a) Selection method: phenotypic and (b) selection method: BLUP-EBV. Journal of Heredity, 2017, Vol. 108, No. 3 325

directional and nondirectional forms of functional epistasis. Griffing (Woolliams 1989; Wray and Hill 1989; Verrier et al. 1993). We also (1960) showed when directional epistasis is present, gametic-phase found that BLUP-EBV generated higher levels of fixation and loss disequilibrium increases the response to directional selection, with of favorable alleles than phenotypic selection. In fact, fixation prob- 22 the response augmented by sσσAA / 2 p , where S is intensity of selec- abilities of favorable mutations depend on the population size; the 2 2 tion σ AA is additive by additive variance and σ p is phenotypic vari- effective size of a population under selection depends on the number ance. This increase in rate of response has been termed the “Griffing of parents selected each generation and the method by which they effect.” Thus, in the presence of directional epistasis, disequilibrium are selected. Since effective population size is reduced more rapidly is on the one hand expected to increase the rate of response, while with BLUP-based selection, the loss of favorable mutations in the it is also expected to decrease the rate of response by decreasing long-term is a concern in breeding programs (Hill 2000). additive genetic variance (the Bulmer effect). Based on a small simulation study, Mueller and James (1983) concluded that if epistatic Fixation and Loss of Favorable QTL variance is small relative to additive variance and the proportion of Favorable allele in our study was defined based on the sign of the pairs showing epistasis is also small, the Bulmer effect dominates the additive effect of an allele. However, in models with gene interaction Griffing effect, and disequilibrium reduces the response to selection favorability can change if the favorability of an allele is determined Downloaded from https://academic.oup.com/jhered/article/108/3/318/2895120 by guest on 23 September 2021 (Walsh and Lynch 2010). by its average effect (or average excess) at the time the measurement We did not observe difference between 2 distributions fitted for is made. In fact, in systems with genetic interactions, selection has QTL effects. Assuming that effects of mutant genes follow a gamma effects that are qualitatively different from selection acting in addi- distribution but their frequencies are independent of their effects tive systems. Selection leads to regular changes in phenotype regard- (i.e., a neutral model), Hill and Rasbash (1986) examined the influ- less of the underlying genetic system. When there is dominance or ence of number and effects of mutant genes on response to selection epistasis, however, the average effects of alleles are changing ran- and variance in response among replicates and found that the shape domly as allele frequencies change. Some alleles may increase in of the distribution of effects of mutant genes on the quantitative trait frequency, only to subsequently decrease in frequency as selection is not usually important, which is in agreement with our findings. proceeds. So, selection in systems with gene interaction leads to ran- In long-term, models including dominance (A/D and A/D/AA) dom shifts in the average effects of alleles (Goodnight 2010). This had lower amount of response. This can be due to the rather fre- will make it difficult to clearly label an allele as “beneficial” as the quent overdominance in this simulation, so loci with positive over- actual effect of the allele will depend on the genetic background, and dominance get stuck at intermediate frequencies. To investigate will change as selection proceeds. whether overdominance may be a causal factor limiting long-term We compared percentage of favorable allele fixation and lost for response to selection, we run model A/D/AA with reduced propor- model A/D/AA with 2 definitions of favorable allele, that is, favora- tion of overdominant loci (Supplementary Figure 5). We compared bility based on the main effect of an allele or its average effect. We response in model A/D/AA when overdominance was not allowed found that if favorable allele was defined as its average effect, then with a scenario in which 20% of loci showed overdominance. When percentage of favorable allele fixation in the last generation was 9% overdominance was not allowed, we found slightly higher response less than when favorable allele was defined as its main effect. In the to selection in long-term compared to a model with overdominant same manner, percentage of favorable allele lost was also 20% higher loci. So, the presence of overdominance might have some effect on when average effect was considered as favorability. Supplementary the cumulated response to selection. Gill (1965a, 1965b) in a simula- Figure 3 shows the changes in average effect of 8 randomly selected tion study showed that over dominance (positive dominance effect) alleles over generations for A/D/AA model (Supplementary Figure 3). reduces selection advance.

Phenotype Versus BLUP Selection Comparison of Simulated Models with Hallander Across all simulated genetic models, compared to phenotypic selec- and Waldmann (2007) tion, selection based on BLUP-EBV accelerated the reduction in VA To compare our results with the findings of Hallander and Waldmann over generations. In fact, compared to phenotypic selection, use of (2007), we simulated their genetic model. When the initial allele fre- family information in BLUP increases the correlation between pre- quency and additive effects were same across all loci, as in their anal- dicted breeding values of relatives and therefore the probability ysis, we obtained similar results and including nonadditive effects of co-selection of close relatives, which in turn leads to increase increased VA in the initial generations (Supplementary Figure 4). in amount of inbreeding and reduction in the effective population However, in our study, VA decreased in all genetic models and includ- size and additive genetic variance (Robertson 1960). This increase ing nonadditive effects did not have effect on the amount of VA at in amount of inbreeding and reduction in additive genetic variance least in short-term. In order to figure out the factors that contrib- explains the observed earlier plateau of response for BLUP than uted to the observed differences between our results and those of phenotypic selection in our results. Also, we found that, phenotypic Hallander and Waldmann (2007), we run their model by changing selection realized higher response in long-term than selection based the parameters assumed in their simulation. The parameters being on BLUP-EBV (results not shown). Compared to long-term response, tested were number of loci, initial distribution of allele frequencies, response to short-term selection of a few generations depends on and distribution of allelic effects. We run 4 different models and in the intensity and accuracy of selection in these generations, how- each model one parameter was changed compared to Hallander and ever, in long-term selection depends on how much genetic variabil- Waldmann (2007) keeping other parameters constant. For example, ity is maintained and how much inbreeding depression occurs over in Model 1, the number of loci affecting the trait of interest was generations, and therefore on effective population size (Robertson increased to 100 rather than 4 loci considered in Hallander and 1960). As BLUP uses family information to increase the accuracy of Waldmann (2007). However, similar to Hallander and Waldmann selection, so BLUP-based selection is expected to give larger short- (2007), the initial allele frequency and loci effects were the same term but smaller long-term responses than phenotypic selection across all loci (for more details see footnote in Supplementary 326 Journal of Heredity, 2017, Vol. 108, No. 3

Figure 4). The results showed that any change in the assumptions of genetic variance in populations that pass through a bottleneck. Evolution. Hallander and Waldmann (2007) would result in depletion of addi- 53:1009–1018. tive variance by directional selection except changing the number Crow JF, Kimura M. 1970. An introduction to population genetics theory. of loci affecting the trait. We found that by increasing number of Caldwell, NJ: The Blackburn Press. Falconer DS, Mackay TFC. 1996. Introduction to quantitative genetics. Har- the loci affecting the trait and keeping other parameters constant, low (UK): Longman. additive genetic variance increased in initial generations similar to Fisher RA. 1918. The correlation between relatives on the supposition of Men- Hallander and Waldmann (2007). However, changes in other param- delian inheritance. Trans R Soc Edinburgh. 52:399–433. eters such as distribution of allelic effects and frequencies resulted in Fuerst C, James JW, Solkner J, Essl A. 1997. Impact of dominance and epistasis reduction of additive variance (Supplementary Figure 4). on the genetic make-up of simulated populations under selection: a model development. J Anim Breed Genet. 114:163–175. Gill JL. 1965a. Effects of finite size on selection advance in simulated genetic Conclusion populations. Aust J Biol Sci. 18:599–617. In the schemes we simulated, additive genetic variance decreased Gill JL. 1965b. Selection and linkage in simulated genetic populations. Aust J Biol Sci. 18:1171–1187. by directional truncation selection, also in presence of nonadditive Downloaded from https://academic.oup.com/jhered/article/108/3/318/2895120 by guest on 23 September 2021 genetic effects. The distribution of QTL effects underlying the trait Goddard ME. 2001. The validity of genetic models underlying quantitative and the presence of nonadditive genetic effects had relatively small traits. Livest Prod Sci. 72:117–127. Goodnight C. 2010. Gene interaction and selection. In: Janick J, editor. Plant effects on the changes in additive variance. Response was relatively breeding reviews: long-term selection: maize, volume 24, part 1. Vol. 1. robust to nonadditive genetic effects in short term, but dominance Hoboken (NJ): John Wiley & Sons, Inc. p. 269–291. decreased long-term response to selection. Goodnight C. 2015. Long-term selection experiments: epistasis and the response to selection. In: Moore JH, Williams SM, editors. Epistasis: meth- Supplementary Material ods and protocols. Springer. Goodnight CJ. 1987. On the effect of founder events on epistatic genetic vari- Supplementary data are available at Journal of Heredity online. ance. Evolution. 41:80–91. Goodnight CJ. 1995. Epistasis and the increase in additive genetic variance— Funding implications for phase-1 of Wrights shifting-balance process. Evolution. 49:502–511. First author benefited from a joint grant from the European Commission Griffing B. 1960. Theoretical consequences of truncation selection based on and Aarhus University within the framework of the Erasmus-Mundus the individual phenotype. Aust J Biol Sci. 13:307–343. joint doctorate “EGS-ABG.” This research was supported in part by Hallander J, Waldmann P. 2007. The effect of non-additive genetic interactions the Danish Strategic Research Council (GenSAP: Centre for Genomic on selection in multi-locus genetic models. Heredity. 98:349–359. Selection in Animals and Plants, contract no. 12-132452). Hansen TF, Wagner GP. 2001. Modeling genetic architecture: a multilinear theory of gene interaction. Theor Popul Biol. 59:61–86. Henderson CR. 1975. Best linear unbiased estimation and prediction under a Acknowledgments selection model. Biometrics. 31:423–447. Hill WG. 2000. Maintenance of quantitative genetic variation in animal breed- We thank anonymous reviewer and Charles Goodnight for useful comments ing programmes. Livest Prod Sci. 63:99–109. and suggestions that significantly improved the article. Hill WG, Bünger L. 2010. Inferences on the genetics of quantitative traits from long-term selection in laboratory and domestic animals. In: Plant breeding References reviews. John Wiley & Sons, Inc. p. 169–210. Barton NH, Turelli M. 2004. Effects of genetic drift on variance components Hill WG, Goddard ME, Visscher PM. 2008. Data and theory point to mainly under a general model of epistasis. Evolution. 58:2111–2132. additive genetic variance for complex traits. PLoS Genet. 4:1–10. Belonsky GM, Kennedy BW. 1988. Selection on individual phenotype and best Hill WG, Rasbash J. 1986. Models of long-term artificial selection in finite linear unbiased predictor of breeding value in a closed swine herd. J Anim population with recurrent mutation. Genet Res. 48:125–131. Sci. 66:1124–1131. López-Fanjul C, Fernández A, Toro MA. 2002. The effect of epistasis on the Bennewitz J, Meuwissen THE. 2010. The distribution of QTL additive and excess of the additive and nonadditive variances after population bottle- dominance effects in porcine F2 crosses. J Anim Breed Genet. 127:171– necks. Evolution. 56:865–876. 179. Lynch M, Walsh B. 1998. Genetics and analysis of quantitative traits. Sunder- Bulmer MG. 1971. The effect of selection of genetic variability. Am Nat. land, MA: Sinauer Associates, Inc. 105:201–211. Mackay TFC. 2001a. The genetic architecture of quantitative traits. Annu Rev Caballero A, Toro MA. 2002. Analysis of genetic diversity for the management Genet. 35:303–339. of conserved subdivided populations. Conserv Genet. 3:289–299. Mackay TFC. 2001b. Quantitative trait loci in Drosophila. Nat Rev Genet. Carlborg O, Haley CS. 2004. Epistasis: too often neglected in complex trait 2:11–20. studies? Nat Rev Genet. 5:618–625. Martinez V, Bunger L, Hill WG. 2000. Analysis of response to 20 genera- Carlborg O, Jacobsson L, Ahgren P, Siegel P, Andersson L. 2006. Epistasis and tions of selection for body composition in mice: fit to infinitesimal model the release of genetic variation during long-term selection. Nat Genet. assumptions. Genet Select Evol. 32:3–21. 38:418–420. Mueller JP, James JW. 1983. Effect on linkage disequilibrium of selection for a Carter AJR, Hermisson J, Hansen TF. 2005. The role of epistatic gene interac- quantitative character with epistasis. Theor Appl Genet. 65:25–30. tions in the response to selection and the evolution of evolvability. Theor Nomura T. 2005. Joint effect of selection, linkage and partial inbreed- Popul Biol. 68:179–196. ing on additive genetic variance in an infinite population. Biom J. Cheverud JM, Routman EJ. 1995. Epistasis and its contribution to genetic 47:527–540. variance-components. Genetics. 139:1455–1461. Pedersen LD, Sorensen AC, Henryon M, Ansari-Mahyari S, Berg P. 2009. Cheverud JM, Routman EJ. 1996. Epistasis as a source of increased additive ADAM: a computer program to simulate selective breeding schemes for genetic variance at population bottlenecks. Evolution. 50:1042–1051. animals. Livest Sci. 121:343–344. Cheverud JM, Vaughn TT, Pletscher LS, King-Ellison K, Bailiff J, Adams E, Perezenciso M. 1995. Use of the uncertain relationship matrix to compute Erickson C, Bonislawski A. 1999. Epistasis and the evolution of additive effective population size. J Anim Breed Genet. 112:327–332. Journal of Heredity, 2017, Vol. 108, No. 3 327

Robertson A. 1960. A theory of limits in artificial selection.Proc R Soc Ser B Walsh B, Lynch M. 2010. Short-term changes in the variance. In: Evolution Biol Sci. 153:235–249. and selection of quantitative traits: I. Foundations. Vol. 2. Sunderland Sorensen DA, Hill WG. 1982. Effect of short-term directional selection on (MA): Sinauer Associates, Inc. genetic-variability—experiments with Drosophila melanogaster. Heredity. Wellmann R, Bennewitz J. 2011. The contribution of dominance to the under- 48:27–33. standing of quantitative genetic variation. Genet Res. 93:139–154. Verrier E, Colleau JJ, Foulley JL. 1993. Long-term effects of selection based Woolliams JA. 1989. Modifications to MOET nucleus breeding schemes to on the animal-model BLUP in a finite population. Theor Appl Genet. improve rates of genetic progress and decrease rates of inbreeding in dairy- 87:446–454. cattle. Anim Prod. 49:1–14. Villanueva B, Kennedy BW. 1990. Effect of selection on genetic-parameters of Wray NR, Hill WG. 1989. Asymptotic rates of response from index selection. correlated traits. Theor Appl Genet. 80:746–752. Anim Prod. 49:217–227. Downloaded from https://academic.oup.com/jhered/article/108/3/318/2895120 by guest on 23 September 2021