SHEEPFEARNET: Sheep fear test behaviors classification approach from video data based on optical flow and convolutional neural networks

Determining the temperament related traits of sheep, such as the coping style with various stress factors such as people, a new environment and social isolation, is essential in terms of improving animal welfare and increasing productivity. The classification of sheep according to their behavioral responses to the mentioned stress factors is evaluated by objective or subjective methods by expert observers. However, visual examinations that rely on human observation are more likely to make mistakes and are time consuming. Therefore, it is important to make this process faster, easier and more reliable. The phenotypic and genetic heritability of temperament traits in sheep are examined using behavioral tests such as arena and isolation box. The spatial features of the temperament classes in these tests are generally similar. At the same time, since the behavior traits are composed of time series, defining or classifying these features with image-based approaches can present challenges. In this study, we propose a video-based approach to overcome this challenge, using videos of behavioral traits obtained from fear tests. In this approach, we used a combination of optical flow for capturing temporal features and convolutional neural networks for capturing spatial features. The experimental results show that, balanced datasets in terms of the number of sheep, the BOF-VGG19 model trained with the transfer learning method is 90%, the BOF-CovnLSTM model using ConvLSTM networks is 95%, and the BOF-CNN model using CNNs is 100%, were determined as the optical flow models that classify fear test behavior traits the most successfully. The success rate of UNB-CNN and B-CNN models trained on raw images was 70%. As a result, we obtained successful results in classifying behavioral traits in models trained with optical flow pre-processed data sets balanced in terms of sheep numbers. At the same time, using a combination of optical flow and convolutional neural networks in videos where spatial features between temperament classes are similar enhanced the classification accuracy of fear behavior traits by capturing temporal features.