Automatic scoring of postures in grouped pigs using depth image and CNN-SVM

• Used an image segmentation method (GrabCut) without capturing background images to extract target objects. • Solved the over-segmentation problem by marking the segmentation area before watershed segmentation. • CNN-SVM model was better than single CNN and SVM models for recognition of multiple postures about grouped pigs. • Realized the automatic recognition of multiple postures about grouped pigs under commercial conditions using depth image. Animal posture is a manifestation of animal behavior, and an animal’s behavior provides information about their health, welfare, and living environment. In recent years, machine vision and machine learning technologies have been widely used to detect individual or group behavior of pigs. The purpose of this study is to use machine vision and deep learning technologies to recognize and score multiple postures (standing, sitting, sternal recumbency, ventral recumbency and lateral recumbency) of pigs under commercial conditions based on depth images. In this study, the Azure Kinect DK depth camera with a top view was used to obtain the depth image of pigs, and the target pig image was obtained by GrabCut image segmentation and watershed segmentation of target object calibration. Then, based on the characteristics of the image, the convex hull, boundary, and the depth distance of the shoulder and the hip were obtained. The ratio of the convex hull perimeter to the boundary and the ratio of the convex hull area to the boundary, as well as the depth distance of the shoulder and the hip, and the depth distance ratio of the shoulder to the hip were obtained as the input of the Convolutional Neural Network-Support Vector Machine (CNN-SVM) classification model, and the model was trained and tested. In various classifier detection experiments, the performance of our pig posture classifier for standing posture and lateral recumbency posture was better, with the area under the receiver operating characteristic (AUC) values being 0.9969 and 0.9967, respectively. However, the performance of sitting posture, sternal recumbency posture and ventral recumbency posture classifier was slightly worse but still had good performance: AUC values were 0.9790, 0.9355 and 0.9795, respectively. The model in this article was used to detect the average postures of pigs in one day (taking the average for eight consecutive days), and it was found that the proportion of lying postures was higher than other postures (lying postures were 72%, standing postures were 20%, and sitting postures were 8%). The proportion of standing postures in the daytime was higher than that in the evening, and lying posture was the opposite. The proportion of the three lying postures also changes over time. This study compared the difference of posture recognition accuracy between the model in this paper (CNN-SVM), SVM and CNN; using the same training data and experimental data, the accuracy of posture recognition of the three models was 94.6368%, 92.2175% and 90.5396%, respectively. Therefore, the recognition accuracy of the model in this paper was improved greatly compared with CNN and SVM.