A reinforcement learning driven two-stage evolutionary optimisation for hybrid seru system scheduling with worker transfer

AbstractAs a new production pattern, the hybrid seru system (HSS) originated from the actual production scenario. In the HSS, the implementation of the worker transfer strategy can further enhance the system's flexibility but is rarely studied at present. In this paper, we develop a reinforcement learning driven two-stage evolutionary algorithm (RL-TEA) to address the hybrid seru system scheduling problem with worker transfer (HSSSP-WT). To conquer this complex problem, the HSSSP-WT is divided into worker assignment-related subproblems (WS) and batch scheduling-related subproblems (BS) according to the problem characteristics. To effectively solve the subproblems, a probability model-based exploration and a lower bound-guided heuristic are presented for the WS, and a greedy search is designed for the BS. Meanwhile, to improve search efficiency and effectiveness, a knowledge-based selection mechanism is proposed to determine which subproblem group to optimise in each generation by fusing a reinforcement learning technique and a lower bound filtering strategy. Moreover, an elite enhancement strategy inspired by the problem property is designed to improve the solution quality. Experimental results demonstrate the effectiveness of the worker transfer strategy and the superior performance of the RL-TEA compared with the state-of-the-art algorithms in solving the HSSSP-WT.KEYWORDS: Hybrid seru system schedulingworker transferreinforcement learninglower bound filteringtwo-stage optimisation AcknowledgmentThis work was supported in part by the National Natural Science Foundation of China under Grant 62273193 and in part by the National Science Fund for Distinguished Young Scholars of China under Grant 61525304.Disclosure statementNo potential conflict of interest was reported by the author(s).Data availability statementThe authors confirm that the data supporting the findings of this study are available within the article.Additional informationFundingThis work was supported by National Natural Science Foundation of China [grant number: 62273193].Notes on contributorsYuting WuYuting Wu received the M.Sc. degree from the Dongbei University, Shenyang, China, in 2020. She is currently pursuing the Ph.D. degree in control theory and control engineering with Tsinghua University, Beijing, China. Her main research interests include intelligent optimisation and seru production system scheduling.Ling WangLing Wang received the B.Sc. degree in automation and the Ph.D. degree in control theory and control engineering from Tsinghua University, Beijing, China, in 1995 and 1999, respectively. Since 1999, he has been with the Department of Automation, Tsinghua University, where he became a Full Professor in 2008. He has authored five academic books and more than 300 refereed papers. His current research interests include computational intelligence-based optimisation and scheduling. He is a recipient of the National Natural Science Fund for Distinguished Young Scholars of China, the National Natural Science Award (Second Place) in 2014, and the Natural Science Award (First Place in 2003, and Second Place in 2007) nominated by the Ministry of Education of China. He is the Editor-in-Chief for the International Journal of Automation and Control, and the Associate Editor for the IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, Swarm and Evolutionary Computation, etc.Jing-fang ChenJing-fang Chen received the Ph.D. degree in control theory and control engineering in 2023 from Tsinghua University, Beijing, China. He is currently a postdoctor in the Department of Automation, Tsinghua University. His main research interests include intelligent optimisation on complex scheduling problems.Jie ZhengJie Zheng received the Ph.D. degree in control theory and control engineering in 2023 from Tsinghua University, Beijing, China. She is currently a Senior Algorithm Engineer with Huawei, Wuhan, China. Her main research interests include the scheduling problem under uncertainty with intelligent optimisation.Zixiao PanZixiao Pan received the B.Sc. degree in automation from the Wuhan University of Technology, Wuhan, China, in 2019. He is currently pursuing the Ph.D. degree in control theory and control engineering with Tsinghua University, Beijing, China. His main research interests include the distributed and green scheduling with intelligent optimisation and reinforcement learning.