A Safe and Self-Recoverable Reinforcement Learning Framework for Autonomous Robots

Reinforcement learning (RL) holds the promise of autonomous robots because it can adapt to dynamic or unknown environments by automatically learning optimal control policies from the interactions between robots and environments. However, the interactions can be unsafe to both robots and environments during the learning phase, which hinders the practical deployment of RL. Some safe RL methods have been proposed to improve the learning safety by using external or prior knowledge to guide safe actions, but it is difficult to assume having this knowledge in practical applications, especially in unknown environments. More importantly, considering failures are unavoidable in practice, current safe RL lacks the capability of recovering to safe states from failures so that the learning cannot be continued and finished. To solve these problems, we propose a safe and self-recoverable reinforcement learning framework that can predict and prohibit other unsafe actions based on known, explored unsafe actions during the exploration process, and can self-recover to a safe state when a failure occurs. The maze navigation simulation results show that our approach can not only significantly reduce the number of failures but also accelerate the convergence of reinforcement learning.