Social mating systems in birds: Resource-defense polygamy–but not lekking–is a macroevolutionarily unstable trait

Abstract Our understanding of the evolution of social mating systems is largely based on an atemporal ecological framework, whereas macroevolutionary and phylogenetic perspectives looking at the causes of mating systems variation are less developed. Here, we present analyses of the evolution of social mating systems in birds at an unprecedented scale, including 66% of the world’s birds and using trait-dependent speciation and extinction models. We found that lekking (no social bond between the sexes) is very rarely lost, in accordance with the hypothesis that a male shifting to investing in one rather than multiple mates would suffer a severe fitness cost. In contrast, resource-defense polygamous lineages (with a weak, transient socio-sexual bond) frequently revert back to monogamy (strong, durable socio-sexual bond) and have an elevated extinction fraction. We tentatively attribute this to the impossibility for females to settle on an optimal parental care strategy under this system. Finally, we found that most gains of lekking have been directly from monogamy, rather than through an intermediate stage of resource-defense polygamy.