Local Selective Vision Transformer for Depth Estimation Using a Compound Eye Camera

A compound eye camera is a hemispherical camera made by mimicking the structure of an insect’s eye. In general, a compound eye camera is composed of a set of single eye cameras. The compound eye camera has various advantages due to its unique structure and can be used in various vision tasks. In order to apply the compound eye camera to various vision tasks using 3D information, depth estimation is required. However, due to the difference between the compound eye image and the 2D RGB image, it is hard to use the existing depth estimation methods directly. In this paper, we propose a transformer-based neural network for eye-wise depth estimation, which is suitable for the compound eye image. We modify the self-attention module with local selective self-attention to take advantage of the compound eye’s hemispherical structure. In addition, we reduce the computational amount and increase the performance through the eye selection module. Using the proposed local selective self-attention and eye selection modules, we are able to improve the performance without large-scale pre-training. Compared to the ResNet-based depth estimation network, our method showed 2.8% and 1.4% higher performance on the GAZEBO and Matterport3D datasets, respectively, with 15.3% fewer network parameters.