Dynamic Attention Model – A Deep Reinforcement Learning Approach for Container Relocation Problem

Container Relocation Problem (CRP) is one of the most important and fundamental problems in the terminal’s operations. Given a specified layout of the container yard with all the container retrieval priorities, CRP aims to identify an ideal container movement sequence so as to minimize the total number of container rehandling operations. In this paper, we are the first to propose a deep reinforcement learning method to tackle the problem. It adopts a dynamic attention model to respond to the changes of the layout. The long short-term memory and multi-head attention layers are introduced to better extract the features of stacks. We use a policy gradient algorithm with rollout baseline to train the model. The experiments demonstrate that our method can solve the problem effectively compared with other classic approaches. We conclude that the deep reinforcement learning approach has a great potential in solving CRP, as it can find desirable solution without using much expert domain knowledge.