Real-time video surveillance on highways using combination of extended Kalman Filter and deep reinforcement learning

Abstract

Highways, as one of the main arteries of transit and transportation in today's world, play a fundamental role in accelerating transportation, and for this reason, continuous monitoring of them is of great importance. Among these, monitoring compliance with transportation laws by vehicles is of utmost importance; for automation, efficient and vehicle-specific models can be used. In this article, a new method for video surveillance of highways is presented using an extended Kalman filter (EKF) and reinforcement learning models. There are three primary stages to the suggested approach. During the first stage, the extended Kalman filter (EKF) is used to identify and track multiple targets. Next, in the second stage, a convolutional neural network (CNN) processes each detected moving item to determine the kind of vehicle. During this stage, the CNN model's ideal configuration is ascertained using a new optimization approach that combines Particle Swarm Optimization (PSO) and reinforcement learning. After identifying the type of vehicle, in the third phase, the proposed method uses a separate CNN model for each target vehicle to assess its compliance with transportation safety principles. It should be mentioned that each vehicle's associated CNN model is configured during this phase using the suggested optimization methodology. Investigations have been conducted into the effectiveness of the suggested method in identifying violations of road safety laws as well as how well it performed in the two phases of vehicle type identification. According to the findings, the suggested approach can identify the kind of vehicle with 98.72% accuracy, which is at least 3.41% better than the approaches that were compared. On the other hand, this model can detect the violation of road safety laws for each vehicle with an average accuracy of 91.5%, which shows at least a 3.49% improvement compared to the other methods.