HKTSG: A Hierarchical Knowledge-guided Traffic Scene Graph Representation Learning Framework for Intelligent Vehicles

Autonomous driving, propelled by data-driven approaches, has made significant advancements. However, challenges remain, particularly in achieving human-like cognitive capabilities within complex traffic scenarios and addressing limitations in model reliability and interpretability. To address these challenges, we draw inspiration from neuroscience and propose the Hierarchical Knowledge-Guided Traffic Scene Graph Representation Learning Framework, denoted as HKTSG. This framework integrates a human-like hierarchical cognition process into a data-driven learning paradigm, leveraging both domain-general and domain-specific knowledge. It systematically learns the visual features of global environments, spatial relations, dynamic interactions among all traffic instances, as well as the distinctive behavioral patterns and intrinsic associations of pedestrians and vehicles. HKTSG facilitates mutual learning across multiple levels and instances, fostering a comprehensive understanding of complex traffic scenarios. Extensive experiments demonstrate the effectiveness and versatility of our framework, consistently achieving state-of-the-art performance across diverse datasets and tasks, including IESG and Non-IESG for the pedestrian collision prediction task, and 571-Honda and 1043-Carla for the subjective risk assessment task.