Surface plasmon-phonon-magnon polariton in a topological insulator-antiferromagnetic bilayer structure

We present a robust technique for computationally studying surface polariton modes in hybrid materials. We use a semiclassical model that allows us to understand the physics behind the interactions between collective excitations of the hybrid system and develop a scattering and transfer matrix method that imposes the proper boundary conditions to solve Maxwell's equations and derive a general equation describing the surface polariton in a heterostructure consisting of N constituent materials. We apply this method to a test structure composed of a topological insulator (TI) and an antiferromagnetic material (AFM) to study the resulting surface Dirac plasmon-phonon-magnon polariton (DPPMP). We find that interactions between the excitations of the two constituents result in the formation of hybridized modes and the emergence of avoided-crossing points in the dispersion relations for the DPPMP. For the specific case of a $\mathrm{Bi}_{2}\mathrm{Se}_{3}$ TI material combined with a 3D AFM such as NiO, $\mathrm{MnF}_{2}$, or $\mathrm{FeF}_{2}$, the polariton branch with low frequency below 2 THz redshifts upon increasing the thickness of TI thin film, which leads to an upper bound on the thickness of the TI layer that will allow an observable signature of strong coupling and the emergence of hybridized states. We also find that the strength of the coupling between the TI and the AFM, which is parametrized by the amplitude of the avoided-crossing splitting between the two polariton branches at the magnon resonance frequency, depends on the magnitude of the magnetic dipole and the linewidth of the magnon in the AFM material as well as on the Fermi energy of Dirac plasmon in the TI. Finally, we show that materials with extremely high quality, i.e., low scattering loss rate, are essential to achieve an experimentally observable strong coupling between a TI and 3D AFM material. The overall analysis identifies the material properties that are necessary to achieve experimentally observable strong coupling for the interaction between THz excitations in a TI/AFM heterostructure and can thereby guide experimental efforts.