SpectraTrack: megapixel, hundred-fps, and thousand-channel hyperspectral imaging

Hyperspectral imaging (HSI) finds broad applications in various fields due to its substantial optical signatures for the intrinsic identification of physical and chemical characteristics. However, it faces the inherent challenge of balancing spatial, temporal, and spectral resolution due to limited bandwidth. Here we present SpectraTrack, a computational HSI scheme that simultaneously achieves high spatial, temporal, and spectral resolution in the visible-to-near-infrared (VIS-NIR) spectral range. We deeply investigated the spatio-temporal-spectral multiplexing principle inherent in HSI videos. Based on this theoretical foundation, the SpectraTrack system uses two cameras including a line-scan imaging spectrometer for temporal-multiplexed hyperspectral data and an auxiliary RGB camera to capture motion flow. The motion flow guides hyperspectral reconstruction by reintegrating the scanned spectra into a 4D video. The SpectraTrack system can achieve around megapixel HSI at 100 fps with 1200 spectral channels, demonstrating its great application potential from drone-based anti-vibration video-rate HSI to high-throughput non-cooperative anti-spoofing.