Bilevel learning for large-scale flexible flow shop scheduling

• Effective and efficient scheduling method for large scale industrial problems. • Bilevel constraint Markov Decision Process to model the scheduling. • A theoretical guarantee of convergence to Stackelberg equilibrium. • Bilevel deep reinforcement learning framework to learn the problem. • Demonstrating the benefits of our method on benchmarks and industrial data. Many industrial practitioners are facing the challenge of solving large-scale scheduling problems within a limited time. In this paper, we propose a novel bilevel scheduler based on constraint Markov Decision Process to solve large-scale flexible flow shop scheduling problems (FFSP). There are many intelligent algorithms proposed to solve FFSP, but they take quite long time to execute or are even not working for large-scale problems. Our scheduler is able to decide the sequence of a large number of jobs in a limited time with the objective to minimize makespans. The upper level is designed to explore an initial global sequence, whereas the lower level aims to look for partial sequence refinements. In the implementation, Double Deep Q Network (DDQN) is used in the upper level and Graph Pointer Network (GPN) lies within the lower level. The two levels are connected by a sliding-window sampling mechanism. Based on datasets from public benchmarks and real-world industrial scenarios with over 5000 jobs, experiments show that our bilevel scheduler significantly outperforms seven baseline algorithms, including three state-of-the-art heuristics, three deep learning based algorithms, and another bilevel model, in terms of makespans and computational time. In particular, it only takes less than 200 s to get solutions of large-scale problems with up to 5000 jobs and matches the performance of the state-of-the-art heuristics.