A Knowledge Distillation Compression Algorithm for Ship Speed and Energy Coordinated Optimal Scheduling Model based on Deep Reinforcement Learning

Ship optimization scheduling, using deep reinforcement learning (DRL), has been extensively researched and implemented. Notably, the deep Q-learning algorithm (DQN) has achieved successful deployment within the optimization scheduling domain. However, there is currently almost no research on compressing and accelerating DQN-based All-Electric Ships (AES) energy scheduling models. This paper proposes a DQN knowledge distillation (DQN-KD) compression algorithm that incorporates the teacher replay memory pool (T-rpm) learning mechanism for compression problem of the DQN-based optimization scheduling model of AES. The DQN-KD algorithm can effectively transfer the knowledge of teacher agent to student agent, and further improve the training efficiency and performance of student agent using the T-rpm learning mechanism. The experimental results conduct on the AES system demonstrate that our proposed compression method is highly effective. Comparing with the teacher model, the Parameters, FLOP and Memory of the student model are significantly reduced by 87.7%, 92.61% and 88.3% respectively. Interestingly, despite these significant reductions, the student agent only experiences a marginal increase of 0.33% in economic consumption compared to the teacher agent. Furthermore, when the Parameters of the student agent are further reduced by 47.5%, and FLOPs by 50.4%, along with a 47.3% reduction in Memory, the resulting increase in economic consumption is only 0.59% compared to the teacher agent. Importantly, even with these notable reductions, the compressed agent maintained strong generalization performance.