SCIRNet: Skeleton-based cattle interaction recognition network with inter-body graph and semantic priority

Cattle are social and sensitive animals that are held together by an intricate social web. Following the inter-action between cattle is of great importance to the producer for effective herd management. To recognize the interaction, computer vision technology has been widely employed and current solutions in the fields are mainly dominated by frame-based approaches using CNNs and skeleton-based approaches using GCNs. Recently, skeleton-based interaction recognition has been gaining increasing attention due to its robustness to learn the representation of behavioral features. One crucial cue in recognizing an interaction is the interactive body parts. In this work, we propose a novel interaction recognition network applicable to cattle called SCIRNet with an inter-body graph generated from the respective skeleton graph of each cattle. More specifically, the inter-body graph enables the network to focus on interactive body parts by connecting some inter-body joints which are considered to be able to represent features of cattle interaction. In addition, we introduce a multi-stream architecture that accounts for relative information between interactive cattle to improve accuracy. In practice, we combine the graph feature with the image feature extracted from the interaction area to extract a visual representation of the interaction area, as well as the semantic priority obtained from our dataset to capture our prior knowledge of the relationship between the action and interaction of cattle. Qualitative and quantitative evaluation evidences the performance of our framework as an effective method to recognize cattle interaction.