Ecological Driving Framework of Hybrid Electric Vehicle Based on Heterogeneous Multi-Agent Deep Reinforcement Learning

Hybrid Electric Vehicles have great potential to be discovered in terms of energy saving and emission reduction, and ecological driving provides theoretical guidance for giving full play to their advantages in real traffic scenarios. In order to implement ecological driving strategy with the lowest cost throughout life cycle in car-following scenario, the safety and comfort, fuel economy and battery health need to be considered, which is a complex nonlinear and multi-objective coupled optimization task. Therefore, a novel multi-agent deep deterministic policy gradient (MADDPG) based framework with two heterogeneous agents to optimize adaptive cruise control and energy management strategy respectively is proposed, thereby decoupling optimization objectives of different domains. Due to the asynchronous of multi agents, different learning rate schedules are analyzed to coordinate learning process to optimize training results. And an improvement on Prioritized Experience Replay technique is proposed, which improves optimization performance of original MADDPG method by more than 10%. Simulations under mixed driving cycles show that, on the premise of ensuring car-following performance, the overall driving cost including fuel consumption and battery health degradation of MADDPG-based method can reach 93.88% of that of DP. And the proposed algorithm has good adaptability to different driving conditions.