This Is the Way: Sensors Auto-Calibration Approach Based on Deep Learning for Self-Driving Cars

The technological advancement of sensors and computational power has opened a new chapter in machine learning for robotics applications, especially in image classification, segmentation, object detection, and self-driving cars. One of the challenges among these applications is improving the systems perception reliability and accuracy through sensors fusion. Hence, the focus on using Stereo-cameras and LiDARs as a complement to its accurate distance measurement. However, the calibration process of the sensors is mandatory before deployment. Some may use the conventional methods, including checkerboards, specific pattern labels, or even human labeling, which is labor-intensive and repetitive as it involves doing the same calibration process every time before using. In this work, we have proposed NetCalib – an auto-calibration methodology based on a deep neural network. This research aims to utilize the power of machine learning to find the geometric transformation between stereo cameras and LiDAR automatically. From the experiments, our method manages to find the transformations from randomly sampled artificial errors and outperforms the linear optimization-based ICP algorithm. Furthermore, this research work is open-sourced to the community to fully use the advances of the methodology and initiate collaboration and innovation in this field.