Vision Fourier transformer empowered multi-modal imaging system for ethane leakage detection

A leak detection is an essential procedure to guarantee reliable functioning during ethane production and transportation with infrared imaging. However, infrared imaging can’t perceive semantic information about objects, such as colors and textures. Visible imaging can provide such information but lacks reliability against bad weather. Multi-modal imaging that utilizes visible and infrared information can be the ultimate solution to compensate for their properties. Thus, this study proposed an innovative multi-modal detection framework, Vision Fourier Transformer-based Ethane Detection (VFTED), to effectively and efficiently fuse visible and infrared information to detect ethane leaks. Specifically, the fast Fourier transform is embedded in the neural network to extract global attention for improved information fusion from VI and IR imaging. Meanwhile, a Fourier multi-layer perceptron (FMLP) is designed to enable neural networks to process complex numbers from Fourier transform. Finally, the fused features are fed into the detector for ethane leak detection. Besides, this article also conveys a new case study to validate the feasibility of the proposed VFTED. Extensive experiments demonstrate the significant improvement brought by the proposed framework over detection’s accuracy and robustness. Hence, the proposed framework enables reliable ethane monitoring with multi-modal imaging.