Collision Avoidance in Pedestrian-Rich Environments With Deep Reinforcement Learning

Collision avoidance algorithms are essential for safe and efficient robot\noperation among pedestrians. This work proposes using deep reinforcement (RL)\nlearning as a framework to model the complex interactions and cooperation with\nnearby, decision-making agents, such as pedestrians and other robots. Existing\nRL-based works assume homogeneity of agent properties, use specific motion\nmodels over short timescales, or lack a principled method to handle a large,\npossibly varying number of agents. Therefore, this work develops an algorithm\nthat learns collision avoidance among a variety of heterogeneous,\nnon-communicating, dynamic agents without assuming they follow any particular\nbehavior rules. It extends our previous work by introducing a strategy using\nLong Short-Term Memory (LSTM) that enables the algorithm to use observations of\nan arbitrary number of other agents, instead of a small, fixed number of\nneighbors. The proposed algorithm is shown to outperform a classical collision\navoidance algorithm, another deep RL-based algorithm, and scales with the\nnumber of agents better (fewer collisions, shorter time to goal) than our\npreviously published learning-based approach. Analysis of the LSTM provides\ninsights into how observations of nearby agents affect the hidden state and\nquantifies the performance impact of various agent ordering heuristics. The\nlearned policy generalizes to several applications beyond the training\nscenarios: formation control (arrangement into letters), demonstrations on a\nfleet of four multirotors and on a fully autonomous robotic vehicle capable of\ntraveling at human walking speed among pedestrians.\n