Polycrystalline Si on SiOx passivating contacts enables some of the highest efficiency single-junction Si photovoltaic devices, but the high-temperature firing process needed for industrial metallization can significantly reduce passivation. We show that after firing, the implied open-circuit voltage, iVoc, for the Al2O3/SiNx/poly-Si/SiOx/c-Si stack is 20–30 mV higher than the SiNx/Al2O3/poly-Si/SiOx/c-Si stack and therefore provides better passivation of the SiOx/c-Si interface. Using effusion measurements and Fourier transform infrared spectroscopy, we demonstrate that more than twice as much hydrogen is retained in the dielectric up to the peak firing temperature of ∼800 °C for Al2O3-capped structures. If the Al2O3 layer is not present in the stack, after firing, the iVoc is lower by 50–100 mV compared to Al2O3/SiNx or SiNx/Al2O3 stacks. These studies will inform future work on the role of dielectrics in aiding the passivation of poly-Si/SiOx passivating contacts.