Reinforcement learning based bilevel real-time pricing strategy for a smart grid with distributed energy resources

The integration of flexible loads, distributed energy resources, and other technologies is becoming common in advance power and energy systems. However, the integration also presents significant challenges due to the increasing complexity and uncertainty. To effectively manage these resources, an adaptive pricing mechanism is needed that can account for their variable availability. Based on this, we propose a new bilevel real-time pricing model that considers different distributed energy resources, uncertainty of renewable energy generation, carbon trading mechanisms, and grid fluctuations. Specifically, the upper-level optimization problem aims to maximize the total profit of the supplier that applies Q-learning algorithm. The lower-level optimization problem addresses the need for each user to make optimal power consumption decisions by constructing an individual Markov Decision Process framework for each user. The bilevel model achieves an effective balance of interests between the supplier and users by simultaneously considering both the upper-level and lower-level optimization problems. Additionally, our model can be efficiently solved using the distributed algorithm without the need to acquire transition probabilities. Simulation results show that the method is highly effective in balancing power supply and demand between the supplier and users, reducing carbon emissions, and mitigating power fluctuations.