Integrated Reinforcement Learning Framework for UAV Swarm Two-Stage Cooperative Multi-Target Detection Tasks

Developing efficient collaborative strategies for UAV swarms is crucial for achieving accurate and rapid execution of the Multi-Target Detecting (MTD) tasks which involve two primary stages in practical scenarios: dispersed search by multi-UAVs in unknown dynamic environments to locate targets, and subsequent aggregation to gather all targets information within the scene, which called detecting and aggregation processes. In recent years, several collaborative strategy methods have been developed for application in UAV swarm mission scenarios. These methods are typically designed for single-stage tasks, and therefore their performance is likely to be suboptimal when applied to multi-stage tasks like the MTD tasks, which have distinctly different characteristics and objectives across stages. In this paper, we propose a novel integrated deep reinforcement learning decision framework that can offer effective collaborative strategies for tasks characterized by distinct stages, denoted as the STDGNet. The STDGNet comprises a Transformer-based Deep Graph Network (TDGN) module alongside two Specialized optimization strategies: the location-dispersion strategy and the cluster-action-consistency strategy. The TDGN module is designed to extract features from observations and interaction dynamics among UAVs, aimed at generating collaborative strategies. The integration of two specialized strategies enables the STDGNet framework to adapt well to multi-stage tasks: during the detection stage, the location-dispersion strategy maintains UAV dispersal to expedite the discovery of more targets; during the aggregation stage, the cluster-action-consistency strategy ensures that UAVs within the same cluster move in the same direction, facilitating the formation of interconnected communication networks. To assess the efficiency and resilience of the proposed framework, we construct an MTD environment where extensive experimentation shows that the STDGNet framework surpasses baseline methodologies.