Abstract The incredible growth of wireless technologies has led to an increase in demand of spectral resources for various communication systems. The allocated spectrum for GSM is insufficient to support applications that operate at high data rates (e.g., multimedia applications or 5G mobile communications). Orthogonal frequency division multiplexing (OFDM) and generalized frequency division multiplexing (GFDM) are the recent technologies with high data rates for multimedia applications and mobile communications. These are the perfect candidates for wireless applications and 5G technologies. In this paper, an in‐depth study was presented where the multicarrier models of orthogonal frequency‐division multiplexing (OFDM) and GFDM are integrated with a sub‐Nyquist sampling architecture. The primary focus is to evaluate and compare their spectrum sensing performance. Also analyzed the performance of both OFDM and GFDM under sub‐Nyquist sampling framework and measured the bit error rate (BER) as a function of the received signal‐to‐noise ratio (SNR).