This page outlines the free tools and datasets the lab disseminating for open science. Researchers and scholars are welcome to use them for advancing knowledge but not for commercial purposes. If you are interested in the full publications of the lab, you can visit the page of “Publications”.
Spatiotemporal Video Encoder Integrated with Metadata Fusion for Action Recognition and Behavior Analysis (Holden et al., 2026)
Automated behavior monitoring and action recognition is challenging in dense, dynamic environments of practical broiler pens, combined with the challenges of handheld smartphone footage. This study developed and validated an integrated pipeline for spatiotemporal action recognition and behavioral analysis of broiler chickens using smartphone-captured video. Our end-to-end framework addresses key technical challenges by first stabilizing raw footage to correct motion artifacts, then employing YOLOv11-Pose for robust detection and pose estimation in crowded scenes, and ByteTrack for multi-object tracking. The core innovation is a novel metadata fusion strategy, where spatiotemporal features from an X3D video encoder were integrated with pose-derived contextual metadata (such as velocity and proximity to feeders and drinkers) using a multilayer perceptron. This intermediate fusion approach enabled the model to disambiguate visually similar but contextually distinct actions, such as drinking versus feeding. The system demonstrated strong performance on five key maintenance behaviors, achieving an average F1-score of 0.864 and an overall test accuracy of 86.41%. The system was particularly effective at detecting drinking and feeding behaviors, with individual F1-scores of 0.949 (94% accuracy) and 0.928 (91.89% accuracy) respectively. The system successfully processed 174 videos, producing valuable insights in the form of behavioral metrics, spatial heatmaps, and individual trajectories. This demonstrates that an integrated system, combining stabilization, advanced multi-task models, and contextual metadata fusion, can effectively use smartphone footage to create a scalable, objective tool for precision livestock farming and welfare research. The key advantages of the proposed framework include its ability to disambiguate visually similar birds and behavior actions through metadata fusion, its use of accessible cost-effective smartphone data, and its balance of accuracy and computational efficiency. The integrated framework can be found via this link: https://github.com/William0Holden/chicken_action_recognition.
Salmonella remains a major cause of foodborne illnesses that are commonly associated with the consumption of contaminated poultry products. Therefore, there is a strong emphasis and a crucial need to reduce Salmonella across the farm to fork continuum. However, there are no easy and rapid on-farm prediction tools to estimate if a flock carries Salmonella. In this study, we used deep learning to develop a Salmonella risk classification model for chicken feces. The model was developed using a two-part framework combining deep learning and tree-based classification techniques. In the first stage, deep learning models such as ResNet152, Xception, VGG19, and others were employed to extract high-level features from chicken fecal images. In the second stage, these features were used by decision tree classifiers including Random Forest and Gradient Boosting for binary classification into Salmonella PCR Positive or Negative. The model was developed using fixed train-validation-test splits on two datasets: an American dataset with 720 images from a controlled multi-serovar Salmonella challenge study and African dataset with 4333 publicly available images. This integrated approach achieved up to 85% classification accuracy when validated across geographical regions. The developed framework was integrated into a user-friendly mobile and web application and allows users to custom train the deep learning classifiers to improve the classification of unseen fecal images. Although the model was primarily validated with the same dataset, this proof-of-concept framework can serve as a decision-support tool for Salmonella control and intervention, contributing to enhanced food safety. Overall, the objective of this study is to develop and validate a hybrid AI approach for Salmonella prediction using fecal images, with explicit cross-region evaluation and proof-of-concept deployment applications. The developed app and machine learning modeling interface can be found here: https://github.com/bvumesh/Salmonella-Classification.
A tactile sensing dataset for whole and cracked eggs (Li et al., 2025)
A total of 377 tactile sensing images were collected, including 173 cracked eggs and 200 whole eggs. The image resolution is 640 x 480 pixels. The same number of bounding box annotations is stored in this repository. The dataset is split into 264 images (70%) for training, 75 images (20%) for validation, and 38 images (10%) for testing. The dataset can be used to train robotic end effectors to grasp eggs and separate the whole and cracked eggs efficiently. Zenodo data link: https://zenodo.org/records/15360469
Monitoring the activity index of animals is crucial for assessing their welfare and behavior patterns. However, traditional methods for calculating the activity index, such as pixel intensity differencing of entire frames, are found to suffer from significant interference and noise, leading to inaccurate results. These classical approaches also do not support group or individual tracking in a user-friendly way, and no open-access platform exists for non-technical researchers. This study introduces an open-source web-based platform that allows researchers to calculate the activity index from top-view videos by selecting individual or group animals. It integrates Segment Anything Model2 (SAM2), a promptable deep learning segmentation model, to track animals without additional training or annotation. The platform accurately tracked Cobb 500 male broilers from weeks 1 to 7 with a 100% success rate, IoU of 92.21% ± 0.012, precision of 93.87% ± 0.019, recall of 98.15% ± 0.011, and F1 score of 95.94% ± 0.006, based on 1157 chickens. Statistical analysis showed that tracking 80% of birds in week 1, 60% in week 4, and 40% in week 7 was sufficient (r ≥ 0.90; p ≤ 0.048) to represent the group activity in respective ages. This platform offers a practical, accessible solution for activity tracking, supporting animal behavior analytics with minimal effort. The interface was published on GitHub (https://github.com/MahtabSaeidifar/AnimalAI, version 1.0, accessed on 1 June 2025) for open access.
Abstract: Automated collection of accelerometer data and subsequent machine learning modeling are prevalent combined methods for animal behavior recognition. However, there is a lack of customized tools for user-friendly machine learning model development. Meanwhile, existing models in previous research could not be directly used for behavior interpretation. The objective of this study was to design and develop a tool for customized machine learning model development and animal behavior analysis using triaxial accelerometer data. A graphical user interface was programmed with Python and saved in a public repository for open access. The interface mainly consists of pages of ‘Manage Project’, ‘Preprocess Data’, ‘Develop Models’, and ‘Analyze Behavior’. An open dataset containing triaxial accelerometer data of six beef cattle was used to test the developed interface. The main results show that users can customize appropriate machine learning models for behavior analytics through several mouse clicks on the interface. A total of 15 models can be selected and trained to determine an optimal one, and model performance can be optimized by adjusting parameters of window size, step size, and training-to-validation ratio. Data imbalance can be solved by merging minority classes into one. The newly developed model has the capacity to analyze overall behavior time budget, statistics (e.g., mean, minimum, maximum, and standard deviation) of each behavior duration, and frequency of behavior sequences. The tool is supportive for automated animal behavior analytics critical to enhancing animal welfare, housing environment, genetics selection, and flock management. GitHub Repository: https://github.com/GuomingLi565/AnimalAccML