TTFNeXt for real-time object detection

Abstract Modern object detectors rarely achieve short training time, fast inference speed, and high accuracy at the same time. To strike a balance among them, we propose single-scale TTFNet and multi-scale TTFNeXt. In this work, we use light-head, single-stage, and anchor-free designs, which enable fast inference speed. Then, we focus on reducing training time and improving accuracy. We notice that encoding more training samples from annotated boxes plays a similar role as increasing batch size, which helps enlarge the learning rate and accelerate the training process. To this end, we introduce a dense regression approach based on Gaussian kernels. We also show through experiments that deformable convolutions in our single-scale detector are not sufficient to handle the scale-variation problem. Therefore, we extend the single-scale detector to a multi-scale version. The multi-scale design will yield redundant detections from different pyramid levels, thus we introduce our cross-level NMS algorithm to efficiently eliminate redundant results. Experiments on MS COCO show that our TTFNet and TTFNeXt have great advantages in balancing training time, inference speed, and accuracy. They can reduce training time by more than three times compared to previous real-time detectors under similar detection accuracy and faster inference speed. When training 120 epochs, our TTFNeXt is able to achieve 33.7 AP/99 FPS and 41.8 AP/40 FPS with single GTX 1080Ti.