Influence of mutation rate on estimators of genetic differentiation--lessons from Arabidopsis thaliana.

Background: The analysis of molecular variation within and between populations is crucial to establish strategies for conservation as well as to detect the footprint of spatially heterogeneous selection. The traditional estimator of genetic differentiation (F(ST)) has been shown to be misleading if genetic diversity is high. Alternative estimators of F(ST) have been proposed, but their robustness to variation in mutation rate is not clearly established. We first investigated the effect of mutation and migration rate using computer simulations and examined their joint influence on Q(ST), a measure of genetic differentiation for quantitative traits. We further used experimental data in natural populations of Arabidopsis thaliana to characterize the effect of mutation rate on various estimates of population differentiation. Since natural species exhibit various degrees of self-fertilisation, we also investigated the effect of mating system on the different estimators.

Results: If mutation rate is high and migration rate low, classical measures of genetic differentiation are misleading. Only Phi(ST), an estimator that takes the mutational distances between alleles into account, is independent of mutation rate, for all migration rates. However, the performance of Phi(ST) depends on the underlying mutation model and departures from this model cause its performance to degrade. We further show that Q(ST) has the same bias. We provide evidence that, in A. thaliana, microsatellite variation correlates with mutation rate. We thereby demonstrate that our results on estimators of genetic differentiation have important implications, even for species that are well established models in population genetics and molecular biology.

Conclusions: We find that alternative measures of differentiation like F'(ST) and D are not suitable for estimating effective migration rate and should not be used in studies of local adaptation. Genetic differentiation should instead be measured using an estimator that takes mutation rate into account, such as Phi(ST). Furthermore, in systems where migration between populations is low, such as A. thaliana, Q(ST) < F(ST) cannot be taken as evidence for homogenising selection as has been traditionally thought.