To tackle the issues of false positives and missed detections arising from inconsistent defect scales and complex, variable background textures in photovoltaic module fault detection, we propose a novel defect detection algorithm based on YOLOv8-AFA. Firstly, an adaptive bottleneck attention mechanism is introduced, which integrates convolutional operations with adaptive average pooling, effectively mitigating the interference caused by complex background textures in photovoltaic modules. Secondly, a multi-scale adaptive fusion mechanism is developed, combining adaptive average pooling, convolution, upsampling, and feature fusion to overcome the challenge of missed detections due to varying defect scales in photovoltaic module fault detection. Finally, an adaptive pooling fusion module is constructed, leveraging both adaptive max pooling and adaptive average pooling to enhance the model's detection capabilities across diverse environments. Experimental results demonstrate that the proposed YOLOv8-AFA algorithm achieves a mean average precision (mAP) of 91.5% in photovoltaic module fault detection tasks, representing a 2.2% improvement over the original YOLOv8 model. Moreover, the generalization capability of the algorithm was rigorously validated on the PASCAL VOC dataset, achieving a mean accuracy of 90.5%, surpassing other methods. This result demonstrates the improved algorithm's generalization performance, providing robust technical support for intelligent fault diagnosis in photovoltaic modules.