Deep segmentation and classification of complex crops using multi-feature satellite imagery

• An end-to-end transferable-learning approach is proposed in this study. • The class-imbalanced problem of traditional methods could be improved. • The improved UNet++ model outperforms other models in late crop growth periods. • The model with multi-feature can provide better spatiotemporal transferability. • The new model is suitable for regions with small differences in crop categories. Accurate remote sensing-based land use and crop maps provide important and timely information for decision support in large-scale agricultural monitoring. Most existing multi-crop products for complex agricultural areas based on traditional machine learning algorithms are not suitable for large agricultural management because of the poor model transfer capabilities. Therefore, a deep segmentation and classification model including spatiotemporal transfer across regions and years must be developed. In this study, a deep learning approach was developed based on the UNet++ architecture by integrating feature fusion and upsampling of small samples for large-scale land use and crop mapping. Classification experiments were conducted for ten categories at four sites using 10 m resolution Sentinel-2A images from 2019 to 2021 containing 4,194,304 pixels. The joint loss, including the label smoothing cross entropy and Dice coefficient, and the mean intersection over union (mIoU) were used to evaluate the model performance depending on different features and upsampling schemes. The joint loss and mIoU values of the training model and the prediction accuracies of the test sites indicate that the scheme including the upsampling and fusion of spectral bands, vegetation indices, and texture features yields the optimal model performance. For comparison, UNet, DeepLab V3+, Pyramid Scene Parsing Network (PSPNet), and random forest models were built. The improved UNet++ model exhibits the best performance, with an overall accuracy, kappa, and macro F 1 above 91 %, 85 %, and 51 %, respectively. The deep segmentation and classification results without training samples demonstrate the spatiotemporal transfer capability of the UNet++ architecture during the key crop growth period. The patch parameter values also indicate that the improved model exhibits a better shape compactness and significantly reduces and suppresses patch fragmentation. Based on the analysis of the overall confusion matrix, the proposed method improves the classification accuracy for datasets with imbalanced land cover and crop types. This study provides a strategy for large-scale and complex land use and crop mapping based on the integration of feature fusion and deep learning.