Identifying plant disease and severity from leaves: A deep multitask learning framework using triple-branch Swin Transformer and deep supervision

Accurate identification of plant disease is of great significance for intelligent agriculture. Currently, plant species, disease, and severity are considered as joint categories in most disease classification methods, which will increase the number of categories and decrease the generalization ability of these models. Compared to disease and severity, information on plant species is less important because different species may suffer from the same disease, and such information is usually known to users. Given this, this paper proposed a novel triple-branch Swin Transformer classification (TSTC) network for classification of disease and severity simultaneously and separately. The TSTC network consists of a multitask feature extraction module, a feature fusion module and a deep supervision module. Firstly, preliminary features are extracted using a triple-branch network, which is built based on Swin Transformer backbone under the multitask classification strategy (i.e., one for disease classification, one for severity classification and the other for deep supervision). After that, these features are fused using compact bilinear pooling technique to enhance the feature extractor’s learning ability and thus more discriminative features can be extracted. Finally, the deep supervision module combines losses from both hidden layers and the last layers of the TSTC so that it can be trained in the direction where all layers can work efficiently for disease and severity classifications. Compared to five widely used classification networks, experiments with the AI Challenger 2018 dataset shows that our proposed TSTC network achieves the highest accuracy with an overall accuracy of 99.00% for disease classification and 88.73% for severity classification.