Resource Allocation With Workload-Time Windows for Cloud-Based Software Services: A Deep Reinforcement Learning Approach

As the workloads and service requests in cloud computing environments change constantly, cloud-based software services need to adaptively allocate resources for ensuring the Quality-of-Service (QoS) while reducing resource costs. However, it is very challenging to achieve adaptive resource allocation for cloud-based software services with complex and variable system states. Most of the existing methods only consider the current condition of workloads, and thus cannot well adapt to real-world cloud environments subject to fluctuating workloads. To address this challenge, we propose a novel Deep Reinforcement learning based resource Allocation method with workload-time Windows (DRAW) for cloud-based software services that considers both the current and future workloads in the resource allocation process. Specifically, an original Deep Q-Network (DQN) based prediction model of management operations is trained based on workload-time windows, which can be used to predict appropriate management operations under different system states. Next, a new feedback-control mechanism is designed to construct the objective resource allocation plan under the current system state through iterative execution of management operations. Extensive simulation results demonstrate that the prediction accuracy of management operations generated by the proposed DRAW method can reach 90.69%. Moreover, the DRAW can achieve the optimal/near-optimal performance and outperform other classic methods by 3 $\sim$ 13% under different scenarios.