Abstract Surface defect detection in industrial manufacturing ensures product quality and prevents malfunctions. To address issues such as multi-scale damage, low contrast, and small defects on the surfaces of industrial components, we propose an efficient multi-scale feature enhancement network (EMFEN) for improving the detection performance of industrial surface defects. First, a multi-scale extraction module is proposed to extract defect features at multiple levels to ensure sufficient semantic information for multi-scale damage and enhance the feature extraction ability of defects with different scales. Dual-orientation attention is then introduced into the detection network to establish a connection between spatial and channel dimensional information, which enables the network to focus on defect regions and filter out background noise. This alleviates the problems of low contrast and small defects. The experimental results confirm that the proposed network demonstrates superior detection performance compared to other detection algorithms across five surface defect datasets. Additionally, the parameters are reduced by 7.9%, the floating-point operations (FLOPs) decrease by 6.7%, and the model size is reduced by 5.2%. These improvements collectively provide an efficient solution for industrial surface defect detection.