To improve electrochemical water splitting, it is essential to develop efficient electrocatalysts. While the incorporation of sulfur atom (S) into nickel–iron electrocatalysts has demonstrated a critical role in boosting oxygen evolution reaction (OER), the detailed mechanism remains ambiguous and elusive. In this study, bottom-up monitoring is employed to deeply investigate the role of sulfur atoms in the OER performance. To achieve this, the nickel–iron oxyhydroxide, after a two-step sulfurization and electrochemical activation process, was converted to activated nickel–iron oxyhydroxide. It was found that the metal–sulfur bond in the new intermediate NiSO4·xH2O underwent reconstruction and phase changes along with enhanced crystallinity, resulting in the appearance of a new metal–oxygen active site. After the reconstruction step, the major phase of the pre-catalyst consisting of Ni(OH)2/FeOOH becomes NiOOH/Fe2O3. The presence of S atoms facilitates the reconstruction of P − NiFeOxHy towards a new catalyst that only requires (242 and 273) mV overpotentials to reach a current density of (10 and 50) mA·cm−2, respectively. This shows a low Tafel slope of 39 mV·dec−1 with long-term durability. Density functional theory (DFT) calculation reveals that the ΔEads values of O* and OOH* on the NiOOH − Fe2O3 are more negative than on the Ni(OH)2–FeOOH, facilitating OER reaction on the former electrocatalyst under the practical test conditions.