In practical industrial applications, the inference speed of deep learning models directly affects the efficiency of industrial production. Therefore, the lightweight real-time detection method of surface defects is an essential task in the industrial process. We need to achieve a favorable balance between efficiency and accuracy since the rising demand for production efficiency. However, most of the existing pixel-level detection methods 1) often adopt huge computational overhead to learn rich features, resulting in slow inference speed and 2) show a performance degradation when applied to different industrial surface defect scenarios. To this end, we propose an efficient targeted design (ETD) for real-time defect detection of surface defects. It consists of two branches: (i) an efficient feature enhancement branch, with global aggregation module (GAM) and cross-scale guide module (CGM) to gradually enhance defect features, and (ii) an edge posterior branch, with verification module (VM) and scale interaction module (SIM) to implicitly guide the boundary details of defects. Specifically, while inheriting this framework, we reconsider the relationship between precision, parameters, and speed so that our model can be applied to different industrial scenarios. Extensive experimental results on four datasets indicate that ETD outperforms other leading saliency detection methods. Meanwhile, our method ETD-S achieves 347 FPS on ESDIs-SOD dataset, 254 FPS on Crack500 dataset, 227 FPS on NRSD-MN dataset and 273 FPS on DAGM dataset. Additionally, we conduct real-time analysis of ETD on an intelligent paradigm for industrial surface defect detection, further demonstrating its efficacy in practical scenarios. ETD demonstrates effective detection performance while achieving a lightweight architecture, which can be implemented using various deep learning frameworks, showcasing substantial potential for real-time surface defect detection. The source code and dataset are publicly available at https://github.com/VDT-2048/ETD.