Real-time Pseudo-LiDAR 3D object detection with geometric constraints

Three-dimension (3D) object detection is an essential task in autonomous driving. Although recent LiDAR-based 3D object detection techniques have been well-studied and achieve high detection accuracy, the cost of LiDAR sensors causes a high premium for their practical implementation. Recently introduced Pseudo-LiDAR based methods that utilize image data to detect objects show great prospects for their high cost-effectiveness, however, they tend to be computational complexity and can not meet the realtime requirement. In this paper, we propose a light-weight Pseudo-LiDAR 3D detection system which achieves both high accuracy and high responsiveness. Specifically, we adopt an efficient depth estimator where Binary Neural Networks (BNN) is employed to achieve timely depth prediction. To tackle the accuracy degradation issue caused by the quantitation of the BNNs, we introduce the geometric constraints of virtual planes into the BNN training to enhance the completeness of the objects and improve their representation in 3D space. For the 3D object detector of our system, we provide effective improving schemes including a deviation-aware (DA) head and a finetuning module (FM) for converting existing LiDAR-based detectors to high efficient Pseudo-LiDAR detection components. Experiments on the KITTI benchmark show that our system can conduct the 3D detection within only 35 ms while achieving competitive results to the state-of-the-art (SOTA) algorithms.