Electrocatalytic CO2 reduction (CO2ER) into formate is a desirable route to achieve efficient transformation of CO2 to value-added chemicals, however, it still suffers from limited catalytic activity and poor selectivity. Herein, we develop a hybrid electrocatalyst composed of bismuth and bismuth oxide nanoparticles (NPs) supported on nitrogen-doped reduced graphene oxide (Bi/Bi2O3/NrGO) nanosheets prepared by a combined hydrothermal with calcination treatment. Thanks to the combination of undercoordinated sites and strong synergistic effect between Bi and Bi2O3, Bi/Bi2O3/NrGO-700 hybrid displays a promoted CO2ER catalytic performance and selectivity for formate production, as featured by a small onset potential of -0.5 V, a high current density of -18 mA/cm2, the maximum Faradaic efficiency of 85% at -0.9 V, and a low Tafel slope of 166 mV/dec. Experimental results reveal that the higher CO2ER performance of Bi/Bi2O3/NrGO-700 than that of Bi NPs supported on NrGO (Bi/NrGO) can be due to the partial reduction of Bi2O3 NPs into Bi, which significantly increases undercoordinated active sites on Bi NPs surface, thus boosting its CO2ER performance. Furthermore, a two-electrode device with Ir/C anode and Bi/Bi2O3/NrGO-700 cathode could be integrated with two alkaline batteries or a planar solar cell to achieve highly active water splitting and CO2ER.