This work reports the preparation of a carbonized SiO2–ZrO2 composite membrane with enhanced H2 permselectivity via a sol-gel process. The sol-gel synthesis of the precursor SiO2–ZrO2-acetylacetonate composite was optimized by studying the effect that the water/alkoxide molar ratio exerted on the final properties of carbon-SiO2-ZrO2. Characterization by thermogravimetry, XRD, and N2 adsorption revealed that utilizing a water/alkoxide molar ratio of 60 was optimal for the retention of carbon and for promoting the qualities of amorphousness and microporosity. Furthermore, using a Si/Zr ratio of 9/1 in a carbon-SiO2-ZrO2 membrane resulted in a high level of H2 permselectivity. This membrane showed H2 permeance of 16 x 10−7 mol m−2 s−1 Pa−1 and H2/N2 and H2/CH4 ideal selectivities of 75 and 148, respectively. More importantly, the H2 permselectivity of the carbon-SiO2-ZrO2 membrane exceeded that of a similarly pore-distributed unmodified SiO2–ZrO2 membrane with an H2 permeance of 3 x 10−7 mol m−2 s−1 Pa−1. This enhanced H2 permselectivity was attributed to the increase in pore volume contributed by ultra-micropore-bearing carbon nanoparticles.