Novel Architecture of Energy Management Systems Based on Deep Reinforcement Learning in Microgrid

In microgrids, energy management systems (EMS) have been considered essential systems to optimize energy scheduling, control and operation for reliable power systems. Conventional EMS researches have been predominantly performed by employing demand-side management and demand response (DR). Nonetheless, multi-action control in EMS is confronted with operational challenges in terms of the profitability and stability. In this paper, energy information systems (EIS), energy storage systems (ESS), energy trading risk management systems (ETRMS), and automatic DR (ADR) are integrated to efficiently manage the profitability and stability of the whole EMS by optimal energy scheduling. The proposed microgrid EMS architecture is optimized by using proximal policy optimization (PPO) algorithm, which has been known to have good performance in terms of learning stability and complexity. A novel performance metric, represented as a burden of load and generation (BoLG), is proposed to evaluate the energy management performance. The BoLG is incorporated into the reward settings for optimizing the management of multi-action controls such as load shifting, energy charging-discharging, and transactions. From the simulation results, it is confirmed that the proposed architecture can improve energy management performance with the proper trade-off between stability and profitability, compared to dynamic programming (DP)-based and double deep Q-network (DDQN)-based operation.