In this paper, a new semantic segmentation algorithm, EU-Net (Efficient U-Net), is proposed to realize surface defect detection of mobile phone screens. Compared with U-Net, the encoder and decoder of EU-Net are modified with EfficientNet-B0 and MBconv Block to enhance the detection efficiency and accuracy. Due to the loss of feature information in the cropping operation, it is removed in our EU-Net to improve the detection accuracy. In addition, conventional image processing techniques are used to enhance the dataset. The experiments are conducted on a dataset collected from a production site of the mobile phone screens to verify the superiority of the proposed algorithm.