Two-Step Diffusion Policy Deep Reinforcement Learning Method for Low-Carbon Multi-Energy Microgrid Energy Management

Coordinately scheduling multi-energy in a power system has attracted great research attention because of the benefits like improved energy utilization efficiency, lower system cost and carbon emission. However, the uncertainties from renewable power generation, energy supply and demand sides make this task highly complex. To solve it, a deep reinforcement learning (DRL) algorithm with a novel diffusion model-based policy is proposed to optimize the problem of energy management in a multi-energy microgrid (MEMG) system. Moreover, a two-step reward function is developed to improve the training performance. To lower the overall carbon footprint and economic costs, the carbon emission trading and green certificate trading market mechanisms are introduced in our system to guide the end-user's energy behaviors. A piece-wise linear carbon price model is proposed to constrain the undesired behavior more strictly and further reduce carbon emissions. The superior performance of the proposed scheduling method is compared with several benchmark methods, i.e., three state-of-the-art DRL algorithms, based on real-world datasets. Numerous case studies including three IEEE standard test systems have illustrated its effectiveness in terms of carbon reduction and cost efficiency, acquiring better convergence speed and stability at the same time, all of which lead our method to a better energy management strategy.