Intelligent Resource Allocation for Edge-Cloud Collaborative Networks: A Hybrid DDPG-D3QN Approach

To handle the ever-increasing IoT devices with computation-intensive and delay-critical applications, it is imperative to leverage the collaborative potential of edge and cloud computing. In this paper, we investigate the dynamic offloading of packets with finite block length (FBL) in an edge-cloud collaboration system consisting of multi-mobile IoT devices (MIDs) with energy harvesting (EH), multi-edge servers, and one cloud server (CS) in a dynamic environment. The optimization problem is formulated to minimize the average long-term service cost defined as the weighted sum of MID energy consumption and service delay, with the constraints of the available resource, the energy causality, the allowable service delay, and the maximum decoding error probability. To address the problem involving both discrete and continuous variables, we propose a multi-device hybrid decision-based deep reinforcement learning (DRL) solution, named DDPG-D3QN algorithm, where the deep deterministic policy gradient (DDPG) and dueling double deep Q networks (D3QN) are invoked to tackle continuous and discrete action domains, respectively. Specifically, we improve the actor-critic structure of DDPG by combining D3QN. It utilizes the actor part of DDPG to search for the optimal offloading rate and power control of local execution. Meanwhile, it combines the critic part of DDPG with D3QN to select the optimal server for offloading. Simulation results demonstrate the proposed DDPG-D3QN algorithm has better stability and faster convergence, while achieving higher rewards than the existing DRL-based methods. Furthermore, the edge-cloud collaboration approach outperforms non-collaborative schemes.