We report an ultra-compact, cost-effective on-chip near-infrared spectroscopy system for CO2 sensing using narrow-band optical filter array based on plasmonic gratings with a waveguide layer. By varying the periodicity of the gratings, the transmission spectra of the filters can be continuously tuned to cover the 2.0 μm sensing window with high spectral resolution around 10 nm. Our experimental results show that the on-chip spectroscopy system can resolve the two symmetric vibrational bands of CO2 at 2.0 μm wavelength, which proves its potential to replace the expensive commercial IR spectroscopy system for on-site gas sensing.