Integrating big data analytics in autonomous driving: An unsupervised hierarchical reinforcement learning approach

In the realm of autonomous vehicular systems, there has been a notable increase in end-to-end algorithms designed for complete self-navigation. Researchers are increasingly applying hierarchical reinforcement learning to autonomous driving tasks to address the rising challenges. This study proposes a novel hierarchical reinforcement learning approach that facilitates the training of sub-networks without the need for manual reward design. First, unsupervised reinforcement learning is used to train skills without predefined rewards. These skills are subsequently integrated as sub-policies in the hierarchical reinforcement learning framework to train the meta-controller, which synthesizes these skills to autonomously execute driving tasks in complex scenarios. This training strategy encapsulates the accumulated knowledge into skills, enhancing the transferability and efficiency of learning for similar tasks compared to traditional reinforcement learning methods. Additionally, compared with general hierarchical reinforcement learning, this method does not need to design rewards for each sub-policy, allowing for an indefinite expansion of diverse sub-strategies, which may potentially yield superior performance in a range of tasks.