Physics-Guided Multi-Agent Deep Reinforcement Learning for Robust Active Voltage Control in Electrical Distribution Systems

Although several multi-agent deep reinforcement learning (MADRL) algorithms have been employed in power distribution networks configured with high penetration level of Photovoltaic (PV) generators for active voltage control (AVC), the impact of the voltage fluctuation of a single PV node on voltage violations of other PV nodes in the network is ignored. Consequently, it leads to the conservativeness of the existing MADRL based AVC algorithms. In this paper, a robust MADRL control algorithm is designed to minimize the nodal voltage violation and line loss with the exploration of coupling voltage fluctuations across all the controlled nodes by coordinating PV inverters, and a physics factor is utilized to guide (physics-guided) the training policy with the expectation of a better performance compared to existing purely data-driven methods. In the proposed physics-guided multi-agent adversarial twin delayed deep deterministic (PG-MA2TD3) policy gradient algorithm, a physics factor, global sensitivity of voltage (GSV), is properly embedded in the algorithm to measure the influence of the nodal voltage fluctuation on voltage violations on the other controlled nodes with PV inverters and this GSV is shared in the learning center to guide the centralized learning and decentralized execution process. The multi-agent adversarial learning (MAAL) embedded with the GSV to seek an adaptive descend gradient for reducing the Q-value function appropriately rather than always assuming the worst case. Therefore, this physics-guided method can reduce the conservation and provide significantly better reward. Finally, the proposed algorithm is compared with several other methods on IEEE 33-bus, 141-bus and 322-bus with three-year data in Portuguese and the results indicate the proposed method can obtain the minimal voltage fluctuation and the best reward in the comparisons.