Scan-free end-to-end new approach for snapshot camera spectral sensitivity estimation

Spectral sensitivity is largely related to sensor imaging, which has drawn widespread attention in computer vision. Accurate estimation becomes increasingly urgent because manufacturers rarely disclose it. In this Letter, we present a novel, compact, inexpensive, and real-time computational system for snapshot spectral sensitivity estimation. A multi-scale camera based on the multi-scale convolutional neural network is first proposed, to the best of our knowledge, to automatically extract multiplexing features of an input image by multiscale deep learning, which is vital to solving the inverse problem in sensitivity estimation. Our network is flexible and can be designed with different convolutional kernel sizes for a given application. We build a dataset with 10,500 raw images and generate an excellent pre-trained model. Commercial cameras are adopted to test model validity; the results show that our system can achieve estimation accuracy as high as 91.35%. We provide a method for system design, propose a deep learning network, build a dataset, demonstrate training process, and present experimental results with high precision. This simple and effective method provides an accurate approach for precise estimation of spectral sensitivity and is suitable for computational applications such as pathological digital stain, virtual/augmented reality display, and high-quality image acquisition.