Parallel Architecture for Low Latency UAV Detection and Tracking Using Robotic Telescopes

This paper presents the implementation of a multi-threaded parallel architecture, which enables telescope-based optical UAV detection and tracking in real-time. For efficient image processing an accurate deep learning object detector is complemented in parallel by a fast object tracker. A transition strategy between detector and tracker is introduced based on the tracker reliability, which improves the object localization accuracy of the system. The deep learning algorithm initializes the tracker and in the subsequent frames the reliability of the tracker is compared to the confidence value of each newly detected object to determine whether a reinitialization is necessary. The implemented architecture successfully demonstrates the parallel combination of an FRCNN detector and a MEDIANFLOW tracker to achieve visual UAV detection and tracking at 100 fps. The proposed reliability-based strategy outperforms a purely detector and tracker-based strategy by 6% and 14% respectively in terms of intersection over union at a threshold of 0.5, in scenarios, when the target UAV is flying in front of a complex background. Additionally, the implemented parallel architecture increases the probability for a flight path estimation, which requires at least two localizations, by 49%, when compared to a non-parallel architecture. Field tests are conducted with the proposed architecture using a telescope system demonstrating UAV detection and tracking at 100 fps in distances up to 4000 m in front of a clear background.