Raising the upper cut‐off voltage of cathode is an effective method to improve the energy density of sodium‐ion batteries (SIBs). However, the high upper cut‐off voltage could cause severe side reactions and injure the cycle life of SIBs as the absence of stable cathode‐electrolyte interface. Some fluorinated co‐solvents have been ever employed and proven effective in stabilizing the cathode‐electrolyte interface to support the normal operation of SIBs under a high upper cut‐off voltage. However, the high‐cost of fluorinated co‐solvents would notably improve battery expenses. In this study, a low‐cost co‐solvent called ethyl methylsulfone (EMS) is introduced into the electrolyte for the Na0.67Mn0.8Cu0.2O2 cathode with a high upper cut‐off voltage of 4.5 V. It is found that a stable and uniform cathode‐electrolyte interface (CEI) forms on the cathode, which mitigates the cathode degradation and enhances the cycling stability of cathode. Consequently, this cathode with the designed electrolyte achieves a high capacity retention of 83.2% after 750 cycles at a current density of 1 C (1 C=110 mAh g‐1). This work provides valuable insights into the development of electrolytes for sodium‐ion batteries working at high‐voltage.