Reconstructing Images Through Multimode Fibers From the Up-Conversion Speckle Patterns via Deep Learning

The mode mixing and mode dispersion in the multimode fiber (MMF) will produce complex speckle patterns in the distal end of the fiber as an object passes through the MMF, rendering image reconstruction to be a challenging task. In recent years, convolutional neural networks have been successfully applied to image reconstruction from speckles. However, the imaging spectra of these studies are mostly in the visible spectrum range and require complete speckle information for reconstruction. In this paper, researchers build an optical imaging system that employs up-conversion imaging technology to collect speckle patterns generated by infrared light transmitting through a multimode fiber and a frequency-doubling crystal. They propose a speckle restoration network (SRNet) based on a generative adversarial network (GAN) to reconstruct speckle images. The generator of GAN uses ResNest and atrous spatial pyramid pooling (ASPP) to extract multi-level features and multi-scale context information, respectively. The discriminator of GAN significantly improves the quality of the reconstructed image generated by the generator. In addition, researchers adopt a special training method named pre-training generator to avoid gradient disappearance or gradient explosion in the training process. With the designed network, high-quality images were successfully reconstructed even with only a portion of the speckle information.