A CNN-Based Real-Time Dense Stereo SLAM System on Embedded FPGA

Simultaneous localization and mapping (SLAM) is the task to estimate agent’s ego-motion in the map and reconstruct the 3D geometric of an unknown environment in parallel. Although many SLAM algorithms have been proposed in the past decades, few efforts have been devoted to conducting accurate real-time dense SLAM on resource- and computation-constrained platforms. In this paper, we leverage a shared binary neural network (BNN) architecture to learn robust feature descriptors for depth estimation and pose estimation modules simultaneously, which not only improves the system’s accuracy, but also reduces the computation cost. Also, we propose several optimization strategies targeting feature extraction, feature aggregation as well as feature matching, and to accelerate them on embedded platform. Experimental results demonstrate that our design maintains accurate real-time pose estimation while yielding high-quality dense 3D maps. Our demo video is available at https://github.com/CICAIsubmission/CICAI2023.