Lightweight graphene-based aerogels have received considerable attention in the field of microwave absorption. Nevertheless, the controlled synthesis of reduced graphene oxide (rGO) aerogels with tailored pore structures for high-performance microwave absorption is a significant challenge. Herein, pore structure manipulation of reduced graphene oxide aerogels has been realized through a freeze-thaw assembly strategy, which involves chemical prereduction, freeze-thaw, further reduction and freeze-dry processes. By adjusting the prereduction time, the ultralight (5.83 mg/cm3) and highly porous (94.9%) rGO-60 aerogel develops a uniform cellular structure that maximizes multiple reflections and scattering of electromagnetic waves among microcellular free spaces. This achieves a minimal reflection loss of −61.63 dB with an effective absorption bandwidth of 7.8 GHz for an ultralow filling ratio (0.74 wt%). The radar cross-sectional (RCS) simulation further confirms that the rGO-60 aerogel can attenuate more electromagnetic energy. Meanwhile, the excellent thermal insulating and anti-frosting properties suggest promising applications in cold and humid environments. This work shows that pore structure engineering is a promising strategy for producing lightweight and multifunctional graphene aerogel-based microwave absorbers for complex environmental applications.