Abstract Aqueous batteries are promising candidates for large‐scale energy storage due to their high safety and low cost. However, the narrow electrochemical stability window (ESW, 1.23 V) and high melting point (0 °C) of water limit the energy density and low‐temperature operation of aqueous batteries. In this work, sulfolane is introduced as a co‐solvent to obtain a water/sulfolane hybrid electrolyte, which participates in the solvated sheath of lithium ions, strengthens the OH bond of water, and disrupts the large‐scale hydrogen bond network. Such an elaborate strategy not only expands the ESW of the hybrid electrolyte to 3.8 V, but also lowers the glass‐transition temperature to −110 °C. As proof of concept, the LiMn 2 O 4 /Li 4 Ti 5 O 12 full batteries assembled by the hybrid electrolyte display high voltage (2.7 V) and excellent low‐temperature performance of 98% capacity retention from 0 to −20 °C, which can even discharge normally and illume an LED light (rated power of 0.02 W, excitation voltage of 1.8 V) at −65 °C.