Optimizing hybrid beamforming in millimeter‐wave massive multiple‐input multiple‐output systems: A gradient projection approach

Abstract Millimeter waves (mmWave) present an enticing opportunity for wireless communication due to their substantial bandwidth. However, mitigating signal losses within this spectrum necessitates employing numerous antennas for transmission and reception. In practical scenarios, dedicating individual RF chains to each antenna is often unfeasible. In response to this limitation, our research investigates a hybrid beamforming approach, seeking to optimize spectral efficiency through an alternating optimization (AO) technique. Our goal is to develop an algorithm that can be easily integrated into diverse hybrid beamforming configurations. On the other hand the pursuit of optimizing spectral efficiency while adhering to the constraints imposed by phase shifters results in a non‐convex problem. To confront this challenge, we employ a gradient descent framework combined with projection methods. We introduce the gradient prediction method (GPM), which leads to a closed‐form solution for projection. Simulations underscore that this hybrid beamforming structure can achieve the performance of a fully digital beamforming method when the number of RF chains is twice the total number of data streams, regardless of the number of antennas involved. Furthermore, we will conduct a comparative performance analysis of our proposed algorithm against other established benchmark algorithms to ascertain its superiority.