DHBP: A dual-stream hierarchical bilinear pooling model for plant disease multi-task classification

• A dual-stream hierarchical bilinear pooling model was proposed. • Models using the approach of different bilinear pooling were investigated. • Optimizing the weights of two tasks using homoscedastic uncertainty. Plant diseases have an important impact on agricultural production and economic efficiency. Timely detection of crop diseases and accurate determination of diseases are important for protecting crop safety and controlling the spread of diseases. Although current plant disease recognition research using deep learning has yielded advanced results, it is difficult to produce good results when applied to the actual plant growth environment. The background of plant disease images acquired under field conditions is complex, and the same crop disease often varies widely due to many uncertainties such as pose, shading, and other factors. Models using fine-grained image recognition methods can extract discriminative fine-grained features, thus enhancing the representation capability of the model. Therefore, in this paper, a dual-stream hierarchical bilinear pooling model is proposed for the multi-task classification of crops and diseases under field conditions with a method for the independent identification of plants and diseases. After optimizing multi-task learning using homoscedastic uncertainty, the plant and disease accuracies of the dataset obtained under field conditions were 84.71% and 75.06%, respectively.