To address the issues of complex algorithm models, poor accuracy, and low real-time performance in the coal industry's coal gangue sorting, a lightweight real-time detection method called YOLOv8s-GSC is proposed based on the characteristics of coal gangue. This method incorporates the ghost module into the YOLOv8s backbone network to reduce the network's parameter count. Additionally, a slim-neck model is used for feature fusion, and a coordinate attention module is added to the backbone network to enhance the network's feature representation capability. The experimental results show: (1) The average precision of the YOLOv8s-GSC model is 91.2%, which is a 0.6% improvement over the YOLOv8s model. The parameters and floating-point computation are reduced by 36.0% and 41.6%, respectively. (2) Compared to other models such as FasterRCNN-resnet50, SSD-VGG16, YOLOv5s, YOLOv7, YOLOv8s-Mobilenetv3, and YOLOv8s-GSConv, the average precision is improved to varying degrees. (3) The YOLOv8s-GSC model achieves a detection speed of 115FPS, meeting the real-time requirements for coal gangue detection. In conclusion, the proposed YOLOv8s-GSC model provides a lightweight, real-time, and efficient detection method for coal gangue separation in the coal industry, demonstrating high practical value.