Epistatic interactions between sex chromosomes and autosomes can affect the stability of sex determination systems

Martijn A. Schenkel*, Leo W. Beukeboom, Ido Pen

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

1 Citation (Scopus)
41 Downloads (Pure)


Sex determination (SD) is an essential and ancient developmental process, but the genetic systems that regulate this process are surprisingly variable. Why SD mechanisms vary so much is a longstanding question in evolutionary biology. SD genes are generally located on sex chromosomes which also carry genes that interact epistatically with autosomes to affect fitness. How this affects the evolutionary stability of SD mechanisms is still unknown. Here, we explore how epistatic interactions between a sexually antagonistic (SA) non-SD gene, located on either an ancestral or novel sex chromosome, and an autosomal gene affect the conditions under which an evolutionary transition to a new SD system occurs. We find that when the SD gene is linked to an ancestral sex-chromosomal gene which engages in epistatic interactions, epistasis enhances the stability of the sex chromosomes so that they are retained under conditions where transitions would otherwise occur. This occurs both when weaker fitness effects are associated with the ancestral sex chromosome pair or stronger fitness effects associated with a newly evolved SD gene. However, the probability that novel SD genes spread is unaffected if they arise near genes involved in epistasis. This discrepancy occurs because, on autosomes, SA allele frequencies are typically lower than on sex chromosomes. In our model, increased frequencies of these alleles contribute to a higher frequency of epistasis which may therefore more readily occur on sex chromosomes. Because sex chromosome–autosome interactions are abundant and can take several forms, they may play a large role in maintaining sex chromosomes.

Original languageEnglish
Pages (from-to)1666-1677
Number of pages12
JournalJournal of Evolutionary Biology
Issue number11
Early online date22-Sept-2021
Publication statusPublished - Nov-2021


  • epistasis
  • female heterogamety
  • male heterogamety
  • sex determination transitions
  • sexual selection & conflicts

Cite this