Multimodal Deep Learning for Crop Yield Prediction

Precision agriculture is a vital practice for improving the production of crops. The present work is aimed to develop a deep learning multimodal model that can predict the crop yield in Ecuadorian corn farms. The model takes multispectral images and field sensor data (humidity, temperature, or soil status) to obtain the yield of a crop. The use of multimodal data is aimed to extract hidden patterns in the status of crops and in this way obtain better results than the use of vegetation indices or other state-of-the-art methods. For the experiments, we utilized multi-spectral satellite images obtained from the google earth engine platform and monthly precipitation and temperature data of the 24 Ecuadorian provinces collected from the Ecuadorian Ministry of agriculture and livestock; likewise, we obtained the area of corn plantation in each province and their corn production for the years 2016 to 2020. Results indicate that the use of multimodal deep learning models (pre-trained CNN for images and LSTM for time series sensor data) gives better prediction accuracy than monomodal prediction models.