Extreme sensitivity of the magnetic ground state to halide composition in FeCl3−xBrx

Mixed halide chemistry has recently been utilized to tune the intrinsic magnetic properties of transition-metal halides---one of the largest families of magnetic van der Waals materials. Prior studies have shown that the strength of exchange interactions, hence the critical temperature, can be tuned smoothly with halide composition for a given ground state. Here we show that the ground state itself can be altered by a small change of halide composition in ${\mathrm{FeCl}}_{3\ensuremath{-}x}{\mathrm{Br}}_{x}$. Specifically, we find a threefold jump in the N\'eel temperature and a sign change in the Weiss temperature at $x=0.08$ corresponding to only $3%$ bromine doping. Using neutron scattering, we reveal a change of the ground state from spiral order in ${\mathrm{FeCl}}_{3}$ to $A$-type antiferromagnetic order in ${\mathrm{FeBr}}_{3}$. From first-principles calculations, we show that a delicate balance between nearest and next-nearest neighbor interactions is responsible for such a transition. These results demonstrate how varying the halide composition can tune the competing interactions and change the ground state of a spiral spin liquid system.