Assessing the intensity of fish feeding activity using fish feeding behavior can help farmers efficiently decide on the amount of feeding bait. However, accurate extraction of fish feeding behavior features is difficult because of the small area of fish in the image and the randomness of fish swimming. To address this problem, an improved MobileNetV3 network, namely multi-scale information fusion (MSIF)-MobileNetV3, was proposed for analyzing the fish feeding behavior. Specifically, MSIF is a novel channel attention module used to replace the Squeeze-and-Excitation (SE) module that improves the attention of the model to fish schools behavior in feeding images using spatial information integration and multi-scale feature fusion. To evaluate the effectiveness of the proposed method, its performance was compared with that of the MobileNetV3 network optimized using multiple training strategies and other classical convolutional neural networks. It was trained and tested using a self-built dataset, and the experimental results showed that the MSIF-MobileNetV3 network using a basic training strategy obtained an optimal classification accuracy of 96.4 % on the test set. Thus, by analyzing the feeding activity of fish, the proposed method can assist in the automatic selection of bait feed under factory farming conditions.