A spatial pyramid pooling-based deep reinforcement learning model for dynamic job-shop scheduling problem

The dynamic job-shop scheduling problem (DJSP) is a typical of scheduling tasks where rescheduling is performed when encountering unexpected events such as random job arrivals and rush order. However, the current rescheduling approaches cannot reuse the trained scheduling policies or the experiences due to the variant size of scheduling problems. In this paper, we propose a deep reinforcement learning (DRL) scheduling model for DJSP based on spatial pyramid pooling networks (SPP-Net). A new state representation is proposed based on the machine matrix and remaining time matrix which is decomposed from the scheduling instance matrix. And a new reward function is derived from the area of total scheduling time where the accumulated reward is negatively linearly dependent with the make-span of a scheduling task. Moreover, a size-agnostic scheduling policy is designed based on the SPP-Net and SoftMax function, which is trained by the proximal policy optimization (PPO). Besides, various paired priority dispatching rules (PDR) are used as available actions. Static experiments on classic benchmark instances show that our scheduling model achieves better results on average than existing DRL methods. In addition, dynamic scheduling experiments are tested and our model obtains better results than the PDR scheduling methods in reasonable time when encountering unexpected events such as random job arrivals and rush order.