Distributed BESS Scheduling for Power Demand Reshaping in 5G and Beyond Networks

The mobile network operators are upgrading their network facilities and shifting to the 5G era at an unprecedented pace. The huge operating expense (OPEX), mainly the energy consumption cost, has become the major concern of the operators. In this work, we investigate the energy cost-saving potential by transforming the backup batteries of base stations (BSs) to a distributed battery energy storage system (BESS). Specifically, to minimize the total energy cost, we model the distributed BESS discharge/charge scheduling as an optimization problem by incorporating comprehensive practical considerations. Then, considering the dynamic BS power demands in practice, we propose a multi-agent deep reinforcement learning (MADRL) based approach to make distributed BESS scheduling decisions in real-time. Particularly, QMIX framework is leveraged to learn the partial policy of each agent in the training phase; while in the execution phase, each BS can make scheduling decisions based on local information. The experiments using real-world BS deployment and traffic load data demonstrate that with our QMIX-based distributed BESS scheduling, the peak power demand charge of BSs can be reduced by more than 26.59%, and the yearly OPEX saving for 2282 5G BSs could reach up to U.S. ${\$}$ 196,000.