The prediction of drug-target affinity (DTA) is crucial for efficiently identifying potential targets for drug repurposing, thereby reducing resource wastage. In this paper, we propose a novel graph-based deep learning model for DTA that leverages adaptive structure-aware pooling for graph processing. Our approach integrates a self-attention mechanism with an enhanced graph neural network to capture the significance of each node in the graph, marking a significant advancement in graph feature extraction. Specifically, adjacent nodes in the 2D molecular graph are aggregated into clusters, with the features of these clusters weighted according to their attention scores to form the final molecular representation. In terms of model architecture, we utilize both global and hierarchical pooling, and assess the performance of the model on multiple benchmark datasets. The evaluation results on the KIBA dataset show that our model achieved the lowest mean squared error (MSE) of 0.126, which is a 0.5% reduction compared to the best-performing baseline method. Additionally, to validate the generalization capabilities of the model, we conduct comparative experiments on regression and binary classification tasks. The results demonstrate that our model outperforms previous models in both types of tasks.