Deep reinforcement learning-based collision avoidance for an autonomous ship

Social interest in autonomous navigation systems for autonomous ships is also increasing. For a robust autonomous navigation system, the location, speed, and direction of the ship and other ships must be identified in real time, and collision avoidance should be performed at an appropriate time by considering the collision risk. In this study, we proposed a collision avoidance method that quantitatively assesses the collision risk and then generates an avoidance path. First, to assess the collision risk, a collision risk assessment method based on the ship domain and the closest point of approach (CPA) was proposed. The ship domain is created with an asymmetric shape considering manoeuvring performance and the COLREGs. The CPA is used to assess quantitative collision risk value. Subsequently, a path generation algorithm based on deep reinforcement learning (DRL) was proposed to determine the avoidance time and to generate an avoidance path complying the COLREGs for the most dangerous ship in terms of collision risk. The information of own ship and target ship such as location, speed, heading, collision risk is used as the input state, and the rudder angle of own ship is set as the output action of the DRL. The cost function related to the path following and the collision avoidance is defined as the reward of the DRL-based collision avoidance method. Additionally, the DRL modes are defined to navigate the flexible avoidance path by changing the ratio between the path following and the collision avoidance. To verify the proposed method, we compared the collision avoidance method with the A* algorithm, which is a traditional path planning algorithm, and analyzed the results for various scenarios. The proposed method reliably avoided collisions through flexible paths for complex and unexpected changes in situations compared to the A* algorithm.