Cost-aware scheduling systems for real-time workflows in cloud: An approach based on Genetic Algorithm and Deep Reinforcement Learning

With the development of cloud computing, a growing number of applications are migrating to a cloud environment. In the process, the real-time scheduling of workflows has gradually become a technical challenge, due to the dynamic and uncertain nature of cloud environments and the complex dependencies between sub-tasks of the workflow. Although various methods have been reported up to now, these methods have their respective shortcomings, such as heuristic-based methods are hard to find optimal scheduling scheme and metaheuristic-based methods incur high computational overhead, which often lead to the violation of QoS (Quality of Service) requirements and increases service renting costs of executing workflows. Inspired by the successful application of Deep Reinforcement Learning (DRL) in cloud job scheduling, this paper proposes a real-time workflow scheduling method which combines Genetic Algorithm (GA) and DRL, aiming to reduce both execution cost and response time. To be specific, we design a real-time workflow scheduling algorithm named GA-DQN by combining the global search capability of GA and the environment awareness decision-making capability of DRL to divides scheduling process into two stages. First, the execution scheme of workflow in virtual machine is calculated when workflow arrives. Then, a DRL agent uses this scheme as the feature of workflow to assign workflow to a suitable virtual machine. In this way, the use of DRL to sense environment increases the computational efficiency of GA, and the execution scheme obtained by GA helps DRL to obtain the feature of workflow. On this basis of real world workflow, three groups of simulation experience are carried out to compare GA-DQN with four baseline method which consist of three traditional methods and a state-of-the-art method. The comparison results demonstrate that GA-DQN outperforms the other methods in terms of response time, execution cost, and success rate across different workloads and cloud instance configurations.