A Reinforcement Learning Approach for Flexible Job Shop Scheduling Problem With Crane Transportation and Setup Times

Flexible job shop scheduling problem (FJSP) has attracted research interests as it can significantly improve the energy, cost, and time efficiency of production. As one type of reinforcement learning, deep Q-network (DQN) has been applied to solve numerous realistic optimization problems. In this study, a DQN model is proposed to solve a multiobjective FJSP with crane transportation and setup times (FJSP-CS). Two objectives, i.e., makespan and total energy consumption, are optimized simultaneously based on weighting approach. To better reflect the problem realities, eight different crane transportation stages and three typical machine states including processing, setup, and standby are investigated. Considering the complexity of FJSP-CS, an identification rule is designed to organize the crane transportation in solution decoding. As for the DQN model, 12 state features and seven actions are designed to describe the features in the scheduling process. A novel structure is applied in the DQN topology, saving the calculation resources and improving the performance. In DQN training, double deep Q-network technique and soft target weight update strategy are used. In addition, three reported improvement strategies are adopted to enhance the solution qualities by adjusting scheduling assignments. Extensive computational tests and comparisons demonstrate the effectiveness and advantages of the proposed method in solving FJSP-CS, where the DQN can choose appropriate dispatching rules at various scheduling situations.