Robot Online 3D Bin Packing Strategy Based on Deep Reinforcement Learning and 3D Vision

In the field of logistics and warehousing, it is extremely challenging and practically useful to realize an intelligent palletizing robot which can quickly stack and place cartons of various sizes in disorder. The core technology is the solution of the Online 3D Bin Packing Problem (Online 3D-BPP). For the task of estimating the size and pose of objects by robots, we propose an object size and orientation estimation algorithm based on Euclidean clustering of point cloud information, principal components analysis and minimum circumscribed matrix fitting. In order to solve the extremely challenging Online 3D-BPP, we propose a solution strategy by combining deep reinforcement learning (DRL) and Monte Carlo tree search (MCTS) algorithm, which can combine the information of the prospective K objects to be packed to find the best packing scheme. At the same time, we use the improved Actor-Critic algorithm to train the model and introduce the packing configuration tree model based on heuristic rules, which overcomes the disadvantage that DRL cannot converge when the degree of discretization of the action space increases. The extensive evaluation demonstrates that our learned policy achieves a more efficient and robust packing strategy than current state-of-the-art methods and is practically usable for real-world applications.