Deep Reinforcement Learning Based Joint Beam Allocation and Relay Selection in mmWave Vehicular Networks

Millimeter-wave (mmWave) can provide abundant spectrum resource in vehicular communication networks. Nevertheless, due to the high path-loss and blocking effects in mmWave propagation, and high mobility of vehicles, downlink services for vehicles would be seriously degraded. In this paper, we firstly propose a deep reinforcement learning-based joint beam allocation and relay selection (JoBARS) scheme to mitigate blocking effects and optimize the total transmission rate of the vehicular network, where the mmWave base station (mmBS) provides multi-user services. When downlinks are blocked, the mmBS can select appropriate idle vehicles as relay nodes to enhance service quality from a global perspective. We set the rate punishment restriction in JoBARS scheme to guarantee each vehicle can obtain high-quality service. Besides, a relaying incentive mechanism (RIM) is proposed to avoid vehicles being overly selected for relaying and ensure that relay vehicles have a higher chance of being served in the next round. We demonstrate that JoBARS scheme can effectively enhance the total transmission rate while alleviating transmission outages caused by severe propagation attenuation of mmWave signals. Compared with Greedy Selection scheme, the total rate and average connection probability of vehicles under JoBARS scheme are nearly 17% and 14% higher when blocking effects are severe.