Comparison of relatedness estimators based on empirical genetic data of humpback whales (Megaptera novaeangliae) in the Gulf of Maine

Relatedness is a measure used in population biology to gain insight into heritability, kin selection, social systems, captive breeding, mating strategies as well as population structure. Several relatedness estimators have been developed to infer pairwise relatedness between individuals from genotype data, and there is considerable interest in their reliability. Relatedness estimators appear to be highly dependent on specific population characteristics (e.g. mating system), and data quality. Therefore, a prior assessment of estimator performance is essential before deciding on which to apply to a specific population and objective. However, the majority of studies aimed at assessing the performance of relatedness estimators have used simulated genotype data, and only rarely empirical genotypes from known pedigrees. The aim of this study was to conduct an evaluation of the performance of the available relatedness estimators using known relationships and empirical genotype data from an outbred population of humpback whales in the Gulf of Maine. Consequently, the most reliable measures can be applied to species with similar characteristics where the relatedness is unknown. We employed data from 20 polymorphic microsatellite markers in 425 individuals and two popular software packages (ML-RELATE, COANCESTRY) to evaluate a suite of common relatedness estimators, categorized as maximum likelihood and method of moments estimators. The results of our study showed that the performance of relatedness estimators was affected by the relationship category targeted in the assessment. The maximum likelihood methods overall performed better, but ambiguities and higher misclassification was detected with relationship categories allowing inherent variance among loci. The necessity of merging the two estimator categories for evaluation is arguable, as method of moments estimators often have values outside the true relatedness range (0,1). Overall, our results indicate that estimates of pairwise relatedness must be conducted with care to avoid incorrect inferences which potentially could have conservation ramifications.