Digital twin-enabled adaptive scheduling strategy based on deep reinforcement learning

The modern complicated manufacturing industry and smart manufacturing tendency have imposed new requirements on the scheduling method, such as self-regulation and self-learning capabilities. While traditional scheduling methods cannot meet these needs due to their rigidity. Self-learning is an inherent ability of reinforcement learning (RL) algorithm inhered from its continuous learning and trial-and-error characteristics. Self-regulation of scheduling could be enabled by the emerging digital twin (DT) technology because of its virtual-real mapping and mutual control characteristics. This paper proposed a DT-enabled adaptive scheduling based on the improved proximal policy optimization RL algorithm, which was called explicit exploration and asynchronous update proximal policy optimization algorithm (E2APPO). Firstly, the DT-enabled scheduling system framework was designed to enhance the interaction between the virtual and the physical job shops, strengthening the self-regulation of the scheduling model. Secondly, an innovative action selection strategy and an asynchronous update mechanism were proposed to improve the optimization algorithm to strengthen the self-learning ability of the scheduling model. Lastly, the proposed scheduling model was extensively tested in comparison with heuristic and meta-heuristic algorithms, such as well-known scheduling rules and genetic algorithms, as well as other existing scheduling methods based on reinforcement learning. The comparisons have proved both the effectiveness and advancement of the proposed DT-enabled adaptive scheduling strategy.