Identifying tomato leaf diseases under real field conditions using convolutional neural networks and a chatbot

• Nearly 9 thousand diseased tomato leaf images were collected in real field condition. • An anomaly detection model was developed to verify if an input image was legit. • YOLOv4 was trained to identify the categories of the tomato diseases in the images. • The models were hosted on a cloud service that allows 24/7 access. • A LINE chatbot was developed as the user interface to access the models. Tomatoes are an essential crop in Taiwan and in many other countries worldwide. Disease is a major threat to tomato production, and disease identification is the first step to limiting production loss. Conventionally, plant disease identification has relied on naked-eye examination in fields by experienced farmers and culture and microscopic examination in laboratories by plant pathologists. However, as of 2021, the tomato industry in Taiwan is facing a labor shortage, and experienced farmers or pathologists are not always available. This study developed a method for the automatic identification of eight tomato disease and pest categories using images of tomato leaves, convolutional neural networks (CNNs), and a chatbot controller. Approximately 9,000 images of tomato leaves affected by 11 diseases and pests were collected in fields or greenhouses. The images and the respective lesions on the leaves were sorted into eight categories according to the appearance of lesions. Three CNNs—an anomaly detection model (ADM), a disease identification model (DIM), and a leaf mold/powdery mildew II distinguishing model (LPDM)—were respectively trained to detect anomalies (i.e., non-leaf images), to sort the images into the eight categories, and to distinguish between leaf mold and powdery mildew II. The three CNNs were hosted on a cloud service. The chatbot controller was programmed to manage the communication between the CNNs and the users through LINE, a mobile instant messaging application. The trained ADM achieved an accuracy of 97.40% in the detection of anomalous images. The trained DIM achieved an accuracy of 93.63% in the categorization of images into the eight tomato disease and pest categories. The trained LPDM achieved an accuracy of 98.70% in the distinction between leaf mold and powdery mildew II. The proposed system can assist farmers in timely identification of tomato leaf diseases and provide simple suggestions for the treatment and prevention of diseases in the identified category.