Data-Driven hierarchical energy management in multi-integrated energy systems considering integrated demand response programs and energy storage system participation based on MADRL approach

In this study, an intelligent and data-driven hierarchical energy management approach considering the optimal participation of renewable energy resources (RER), energy storage systems (ESSs) and the integrated demand response (IDR) programs execution based on wholesale and retail market signals in the multi-integrated energy system (MIES) structure is presented. The proposed objective function is presented on four levels, which include minimizing operating costs, minimizing environmental pollution costs, minimizing risk costs, and reducing the destructive effects of cyberattacks such as false data injection (FDI). The proposed approach is implemented in the structure of the central controller and local controller and is based on the multi-agent deep reinforcement learning method (MADRL). The MADRL model is formulated based on the Markov decision process equations and solved by multi-agent soft actor-critic and deep Q-learning algorithms in two levels of offline training and online operation. The different scenario results show operation cost reduction equivalent to 19.51%, risk cost equivalent to 19.69%, cyber security cost equivalent to 24%, and pollution cost equivalent to 20.24%. The proposed approach has provided an important step in responding to smart cities challenges and requirements considering advantage of fast response, high accuracy and also reducing the computational time and burden.