Joint Optimization for Cooperative Service-Caching, Computation-Offloading, and Resource-Allocations Over EH/MEC-Based Ultra-Dense Mobile Networks

Mobile edge-computing (MEC) enabled ultra-dense networks (UDNs), which merges edge-computing with UDNs, can provide enormous benefits, e.g., ultra-low latency. However, due to the ultra-dense deployment of small-cell base stations (SBSs), it becomes infeasible to just depend on the grid power for energy providing, and also it is challenging to jointly optimize service-caching, computation-offloading, and resource-allocation. Integrating energy-harvesting (EH) techniques into MEC-enabled UDNs, we investigate the joint optimization for cooperative service-caching, computation-offloading, and resource-allocation. In our considered UDNs, there exist a large number of EH-based mobile users (MUs) and a mixture of on-grid SBSs, powered by electric grid, and off-grid SBSs, powered by solar, radio frequency (RF) energy, etc. We formulate an energy minimization problem to minimize the sum of weighted energy consumption of all MUs and off-grid SBSs. Also, we develop a two-timescale based joint cooperative service-caching, computation-offloading, and resource-allocation scheme based on the hierarchical multiagent deep reinforcement learning (HMDRL). Using HMDRL, we first derive SBSs' cooperative service-caching policies which are updated in each time frame consisting of multiple time slots. Then, we derive MUs' and SBSs' computation-offloading policies and SBSs' computation resource-allocation policies, which are updated in each time slot. Finally, we validate and evaluate the performances of our proposed schemes through simulations.