Pedestrian Crossing Prediction Based on Invariant Feature Extraction of Cross-Spectral Images

Pedestrian crossing prediction is one of the significant components in the decision-making in the autonomous driving system. And the accuracy of its prediction is to decide whether the autonomous vehicle can forecast the future location of pedestrians, so as to avoid hitting. However, RGB data is easily affected by the environment. When the illumination is insufficient, it is difficult to extract obvious pedestrian features to judge whether the pedestrian has the intention to cross, while we know far-infrared image is robust to the imaging illumination conditions. Therefore, we introduce pseudo-far-infrared data generated by a pre-trained CycleGAN model as a supplement to form a cross-spectral pedestrian crossing prediction framework. And according to the local context features of pedestrians and pedestrian motion information in RGB and pseudo-far-infrared for pedestrian crossing prediction. In order to validate the usefulness of our method, In order to verify the prediction performance of our method, we carried out comparative experiments on two public driving datasets JAAD and PIE. According to the detailed analysis of the experimental results, it can be considered that our proposed method has a promoting effect on pedestrian crossing prediction.