67 Precision identification and weight assessment of cattle using supervised machine learning on body surface keypoints

Abstract Computer vision systems (CVS) offer identification solutions for animals with distinct coat patterns, but are less effective for solid-colored herds. In addition, they can be used to measure critical phenotypes, such as body weight (BW). Providing both BW and identification for solid-colored animals can help farmers make decisions. This study aimed to 1) develop an automatic CVS capable of identifying using the Euclidean distance between keypoints located at specific anatomical landmarks (e.g., bony prominences), and 2) predict the BW using features extracted from these keypoints. The keypoint model was trained to identify seven keypoints per cow, located at the left and right hips, left and right pin bones, tail head, sacral, and cervical vertebrae. These locations remain constant as body condition score of a cow changes. The Euclidean distance between these keypoints was used to generate biometric features for each cow. The keypoint detection model was trained using 3,928 images over 900 epochs with a batch size of 14 and validated using 391 images. This trained model was used to predict keypoints on a different set of 41 cows over 5 d, resulting in a total of 6,944 images which correspond to the dataset for cow identification. The Random Forest (RF) algorithm for cow identification was trained with the first 4 d (5,376 images), and the last day served as the test set (1,568 images). For the cow identification, a RF algorithm was employed on the Euclidean distance of the keypoints. The hyperparameters were determined through a grid search on the training set, utilizing 5-fold cross-validation. The hyperparameters ‘mtry’ (which ranged from 1 to 10) and ‘ntree’ (which ranged from 250 to 500, in increments of 50) were optimized. For the second objective, due to the completely new environment, the keypoint model was fine-tuned using transfer learning for 240 epochs with a batch size of 14, utilizing 310 images. The model predicted keypoints in 1,593 images from different beef-on-dairy animals. This dataset was used to train the BW prediction model. The Euclidean distances from the predicted images and sex were used as features. A RF model was trained using a leave-one-animal-out cross-validation approach. The hyperparameters were determined through a grid search on the training set, using 10-fold cross-validation. The ‘mtry’ parameter ranged from 2 to 16, in increments of 2. For animal identification, the model achieved an accuracy, precision, recall, and F1-score of 92.7%, 89.0%, 90.2%, and 92.7%, respectively. To predict BW, the RF model achieved an R² of 0.86 and a root mean squared prediction error of 36.9 kg, representing 6.7% of the observed values average. These results suggest that keypoints located on the dorsal body surface can identify and weigh individual animals, even those lacking distinct coat color patterns.