An Intelligent Energy Management Strategy for Hybrid Vehicle with irrational actions using Twin Delayed Deep Deterministic Policy Gradient

Abstract Deep reinforcement learning (DRL) is a promising approach to establish energy management strategies (EMSs) for hybrid electric vehicles and how to deal with physical constraints is one of the key challenges in DRL-based EMSs. This paper established a DRL-based EMS for a power-split hybrid electric vehicle and a novel reward function with punishments on irrational actions is proposed based on a rough model of the vehicle. Twin Delayed Deep Deterministic Policy Gradient (TD3), which could enhance the capability of dealing with complicated tasks than the widely-used Deep Deterministic Policy Gradient (DDPG), is adopted to enable online optimization of the EMS. Simulation study with the vehicle model has been conducted to compare the two algorithms using different tasks. In contrast to the DDPG-based EMS, the TD3-based one can obtain the optimal policy with the computation time reduced by 10% and the fuel consumption by 7.28% in complicated tasks including physical constraints recognition and avoiding. Whereas in the simple tasks irrespective of constraints, similar performance has been achieved. It is demonstrated that TD3 is more suitable for complicated tasks and more robust than DDPG.