Successfully commercialized poly(ethylene oxide) (PEO)-based solid polymer batteries (SPBs) are expected to continuously play a key role in the next generation of high-energy density lithium-ion battery technologies. However, the introduction of high-voltage cathodes, accompanied by safety concerns such as PEO decomposition and the associated gas release, is worthy of more attention. This study employs in situ DEMS to study the gassing behavior of LiCoO2|PEO-LiTFSI|Li SPBs. The experiments, together with theory calculations, reveal that a surface catalytic effect of LiCoO2 is the root cause of the unexpected H2 gas release of PEO-based SPBs at 4.2 V. The surface coating of LiCoO2 with a stable solid electrolyte Li1.4Al0.4Ti1.6(PO4)3 (LATP) can mitigate such a surface catalytic effect and therefore extend the stable working voltage to >4.5 V. The crossover effect of HTFSI, which is generated at the cathode side due to oxidation/dehydration of PEO and reacts with lithium at the anode side, is proposed to explain the H2 generation behavior.