Helmet wearing is a major concern for the safety and protection of people on the construction site. Statistic data demonstrate that injuries and accidents occur mainly due to not following prescribed procedures, i.e., not wearing helmet. Camera-based surveillance system can conduct online monitoring task to detect such abnormalities through captured images with image processing system analysis. Although deep learning-based method can achieve higher image identification performance, it requires extensive hardware support of the computational resources. Therefore, it is imperative to design a lightweight network with lower hardware requirement to address such problem. In this paper, a GhostNet, YOLOv5 and a lightweight network are combined to design a model to analyze the image for online monitoring with faster processing speed. The performance of the proposed model is compared with those of the mainstream lightweight models. Experimental results have demonstrated that the proposed model has higher detection accuracy and flexible adaptability.