This paper addresses the challenge of low accuracy in wave height prediction due to inadequate consideration of irregular topological structures and spatio-temporal dependencies of wave measurement points. We propose a novel spatio-temporal wave prediction method, named the Spatio-Temporal Graph Wave Network (STGWN). This model employs graph convolution instead of conventional 2D convolution, enabling spatial domain modeling of wave fields on a graph structure. Simultaneously, it utilizes LSTM for temporal modeling, effectively enhancing the representation of complex dynamic spatio-temporal correlations among wave nodes. Additionally, a dynamic wave position-aware mechanism is constructed to better identify feature variations among different wave nodes. Furthermore, a multi-scale feature fusion mechanism is designed within the model to prevent the risk of crucial temporal feature disappearance during the extraction of spatial dimension features. The proposed spatio-temporal method for Significant Wave Height (SWH) prediction is validated on a wave dataset comprising four different node distribution characteristics within the study area. Results demonstrate that the STGWN model outperforms comparative models in terms of prediction accuracy and stability across multiple evaluation metrics in the four experimental setups.