Reentrant hybrid flow shop scheduling with stockers in automated material handling systems using deep reinforcement learning

In smart factories, automated material handling system (AMHSs) replace manual material handling to increase production efficiency, in which stockers serve as the temporary storage for work-in-process inventories. Furthermore, costly machines and complex processes in advanced manufacturing lead to the need for multiple reentrant processes of the same workstation. However, previous works on scheduling problems rarely considered the function of stockers and reentrant processes, and their approaches were mostly based on metaheuristic algorithms. Recent advances in artificial intelligence enable the possibility of solving scheduling problems using deep reinforcement learning (DRL). Therefore, this work investigates the reentrant hybrid flow shop scheduling problem with stockers (RHFS2) inspired by AMHSs, and solves it by DRL. Firstly, the states, actions, and rewards of the Markov decision process for this problem are designed; and then, two deep Q network (DQN) approaches based on the actions for determining machines and jobs, respectively, are proposed. Simulation results demonstrate that our proposed DQN approaches outperform for finding better solutions of different-scale problems than classical metaheuristic algorithms.