Autoencoder Neural Network-Based STAP Algorithm for Airborne Radar with Inadequate Training Samples

Clutter suppression is a key problem for airborne radar, and space-time adaptive processing (STAP) is a core technology for clutter suppression and moving target detection. However, in practical applications, the non-uniform time-varying environments including clutter range dependence for non-side-looking radar lead to the training samples being unable to satisfy the sample requirements of STAP that they should be independent identical distributed (IID) and that their number should be greater than twice the system’s degree of freedom (DOF). The lack of sufficient IID training samples causes difficulty in the convergence of STAP and further results in a serious degeneration of performance. To overcome this problem, this paper proposes a novel autoencoder neural network for clutter suppression with a unique matrix designed to be decoded and encoded. The main challenges are improving the accuracy of the estimation of the clutter-plus-noise covariance matrix (CNCM) for STAP convergence, designing the form of the data input to the network, and making the network successfully explored to the improvement of CNCM. For these challenges, the main proposed solutions include designing a unique matrix with a certain dimension and a series of covariance data selections and matrix transformations. Consequently, the proposed method compresses and retains the characteristics of the covariances, and abandons the deviations caused by the non-uniformity and the deficiency of training samples. Specifically, the proposed method firstly develops a unique matrix whose dimension is less than half of the DOF, meanwhile, it is based on a processing of the selected clutter-plus-noise covariances. Then, an autoencoder neural network with l2 regularization and the sparsity regularization is proposed for the unique matrix to be decoded and encoded. The training of the proposed autoencoder can be achieved by reducing the total loss function with the gradient descent iterations. Finally, an inverted processing for the autoencoder output is designed for the reconstruct ion of the clutter-plus-noise covariances. Simulation results are used to verify the effectiveness and advantages of the proposed method. It performs obviously superior clutter suppression for both side-looking and non-side-looking radars with strong clutter, and can deal with the insufficient and the non-uniform training samples. For these conditions, the proposed method provides the relatively narrowest and deepest IF notch. Furthermore, on average it improves the improvement factor (IF) by 10 dB more than the ADC, DW, JDL, and original STAP methods.