First-principles study of magnetic states and the anomalous Hall conductivity of MNb3S6 (M=Co,Fe,Mn, and Ni)

Here, inspired by the observation of the extremely large anomalous Hall effect in the absence of applied magnetic fields or uniform magnetization in CoNb₃S₆ [Nat. Commun. 9, 3280 (2018); Phys. Rev. Research 2, 023051 (2020)], we perform a first-principles study of this and related compounds of the MNb₃S₆ type with different transition metal M ions to determine their magnetic orders and the anomalous Hall conductivity (AHC). We find that noncoplanar antiferromagnetic ordering is favored relative to collinear or coplanar order in the case of M = Co, Fe, and Ni, while ferromagnetic ordering is favored in MnNb₃S₆ at low temperatures. The AHC in these materials with noncoplanar spin ordering can reach about e²/h per crystalline layer, while being negligible for coplanar and collinear cases. We also find that the AHC depends sensitively on doping and reaches a maximum for intermediate values of the local spin exchange potential between 0.3 and 0.8 eV. Our AHC results are consistent with the reported Hall measurements in CoNb₃S₆ and suggest a possibility of similarly large anomalous Hall effects in related compounds.