Industrial product surface defect detection is a challenging task, In this paper, we take flange surface defect detection as the research target and propose a high-performance target detection framework to solve some problems in flange surface defect detection, such as inconspicuous features, small target scale and irregular morphology. The proposed model is based on the You Only Look Once (YOLOX) algorithm for improvement. The improved network architecture makes the model more sensitive to the details of the target by increasing the output of the backbone feature extraction network. The extraction and fusion of features is enhanced by using RFP (Recursive Feature Pyramid) and CBAM (Convolutional Block Attention Module) in the neck of the network. In addition, we compare the effect of different single data enhancement methods on the training effect and propose an enhancement method for flange surface defect data to address the problem of low defect data. Experiments show that the flange surface defect detection algorithm proposed in this paper balances accuracy and speed, outperforms existing advanced detection models, and has good detection capability.