Speciation patterns of related species under the hybrid zone: A case study of three sclerophyllous oaks in the east Himalaya–Hengduan Mountains

Abstract Speciation is often accompanied by frequent gene exchanges, which have been gradually recognized as a common phenomenon in nature. Although gene flow may influence different reproductive isolations, the specific mechanism of the process still requires more experimental evidence, especially in hybrid populations that have no significant differentiation and isolation. To address this challenge, this study aims to elucidate the underlying mechanisms of sympatry and parapatry in related species. Three sclerophyllous oaks ( Quercus spinosa , Quercus aquifolioides and Quercus rehderiana ), which are mainly distributed in the sympatry/parapatry of the East Himalaya–Hengduan Mountains and adjacent regions, were used to explore the population dynamics and evolutionary history. Based on 12,420 genome‐wide single nucleotide polymorphism datasets, gene flow detection showed that the three species did not present any obvious genetic barriers. Evolutionary analysis indicated that the three species diverged during the Tertiary Period, and no migration events occurred in the early stages of species divergence. Combined with the data of 19 ecological factors, geological movements and climatic turbulence caused the rapid radiated differentiation of the three species during the Neocene, and similar selective pressures resulted in the same evolutionary pattern based on demographic history analysis. In addition, the predicted niche occupancy profiles and Generalized Dissimilarity Modelling revealed that the three species occupied distinct niches and had significant differences in ecological adaptation, which may explain the specific morphological characteristics of the different species. Therefore, we believe that the populations of the three related species underwent adaptive evolution in different habitats during the early stages of divergence. This study provides new experimental evidence of the formation patterns of parallel speciation.