Caching and Computing Resource Allocation in Cooperative Heterogeneous 5G Edge Networks Using Deep Reinforcement Learning

In this work, we explore a framework for a 5G non-standalone (NSA) heterogeneous network, to meet heterogeneous content requests for users moving in vehicles. We consider that an enhanced NodeB (eNB) acts as a macrocell and next-generation NodeBs (gNBs) act as the small cells. To reduce the downstream latency, entire (or part) of the popular contents are fetched from the core network and cached (stored) at the eNB and gNBs. The computing resources are required at the eNB and gNBs along with the caching resources, for content compression and decompression, leading to a reduced requirement for the caching resources. The eNB and gNBs cooperatively decide on the resources (caching and computing) to be allocated. In this network planning approach, first we compute the optimal coverage radius of the eNB and gNBs. Thereafter, we identify the optimal number of non-overlapping gNBs under the coverage area of the eNB. Finally, we propose a novel deep-Q network (DQN)-based algorithm to train the centralized controller agent so as to identify an optimal policy for caching and computing resource allocation. In case the content popularity and road traffic condition change, the agent can be trained again so as to identify a new optimal policy. We also explore the resource allocation policy using other optimization techniques, such as pattern search, genetic algorithm, and multi-start search. The proposed DQN-based algorithm is scalable and shows an average percentage gain of 66.52%, 76.31%, and 53.64% in terms of computation time to identify an optimal policy for caching and computing resource allocation, over pattern search, genetic algorithm, and multi-start search technique, respectively.