Coal gangue sorting is an essential process in coal mine production, and automatic identification of coal gangue is a prerequisite for intelligent sorting. Coal gangue object detection based on deep learning can make full use of image features, and the upper limit of accuracy is high, but there are problems such as large amount of model calculation and difficulty in deploying real-time sorting sites. Aiming at the above problems, a lightweight DVBS-YOLOv8s coal gangue object detection method was proposed. To enhance the inference speed of the model, DSConv and Vanillanet modules were introduced into the backbone of YOLOv8s, and Slim-neck was incorporated into the neck of the network to achieve lightweight design. Additionally, to improve the feature capture ability of the model for conveyor belt background and coal gangue objects, BoT and Slim-neck were introduced. Finally, WIoU and Soft-NMS algorithms were used to enhance the training effect. An image acquisition test platform was set up, and a dataset of 14 790 coal gangue images was self-constructed for testing. The results show that in the coal gangue object detection task, compared with YOLOv8s, the improved DVBS-YOLOv8s-WS network model pays more attention to global features, increases the receptive field, and inference speed increase by 32%. Compared to the YOLOv5 series, YOLOv7 series, SSD, Faster-RCNN, and YOLOv8s, both detection accuracy and speed are improved, providing reference for achieving high-precision real-time sorting of coal gangue based on deep learning.