Anisotropic exchange coupling and ground state phase diagram of Kitaev compound YbOCl

Rare-earth chalcohalide $\mathrm{RECh}X$ (RE = rare earth; Ch = O, S, Se, Te; $X$ = F, Cl, Br, I) is a newly reported family of Kitaev spin liquid candidates. The family offers a platform where a strong spin-orbit coupling meets a van der Waals layered and undistorted honeycomb spin lattice, which outputs highly anisotropic exchange couplings required by the Kitaev model. YbOCl is the first single crystal of the family we grew, with a size up to $\ensuremath{\sim}15\phantom{\rule{0.16em}{0ex}}\mathrm{mm}$. We have performed magnetization and high magnetic field electron spin resonance measurements from 2 to 300 K. We develop the mean-field scenario for the anisotropic spin system, with which we are able to well describe the experiments and reliably determine the fundamental parameters. The self-consistent simulations give the anisotropic spin-exchange interactions of ${J}_{\ifmmode\pm\else\textpm\fi{}}$ ($\ensuremath{\sim}\ensuremath{-}0.3\phantom{\rule{0.16em}{0ex}}\mathrm{K}$) and ${J}_{zz}$ ($\ensuremath{\sim}1.6\phantom{\rule{0.16em}{0ex}}\mathrm{K}$), and $g$ factors of ${g}_{ab}$ ($\ensuremath{\sim}3.4$) and ${g}_{c}$ ($\ensuremath{\sim}2.9$). Based on the spin-exchange interactions, we employ the exact diagonalization method to work out the ground state phase diagram of YbOCl in terms of the off-diagonal exchange couplings. The phase diagram hosting rich magnetic phases including the spin-disordered one, sheds light on the novel magnetic properties of the family, particularly the Kitaev physics.