Energy Management of Hybrid Electric Vehicles via Deep Q-Networks

The design of the supervisory energy management strategy for a Hybrid Electric Vehicle (HEV) has a significant influence on the potential fuel economy gains and on the amount of calibration required to achieve acceptable performance on a variety of real-world routes. Among different design methods, the Equivalent Consumption Minimization Strategy (ECMS) allows one to convert the global optimization problem into a one-step optimization that is suitable for online implementation, through the introduction of a tunable equivalency factor. However, in order to keep the battery State of Charge (SoC) bounded when the drive cycle is not known a priori, ECMS-based energy management strategies typically require significant calibration efforts. This paper presents an automated development process for the energy management strategy for a mild HEV using a Deep Reinforcement Learning (DRL) algorithm, which leverages a database of simulated real-world routes. The policy learned by the DRL agent is compared against the result of deterministic energy management strategies, specifically Dynamic Programming and Adaptive ECMS, showing very similar characteristics.