Developing AI to monitor changes in social behaviour for the early detection of disease in dairy cattle

Daria Baran, Axel Montout, Jing Gao, Marco Ramirez Montes de Oca, Richard Bruce, Asheesh Sharma, Phoenix Yu, Kira Clements, Tony Fang, Huimin Liu, Ben Lecorps, Mike Mendl, Kristen Reyher, Siobhan Mullan, Tilo Burghardt, Neill Campbell, Suzanne Held, Daniel Enriquez-Hidalgo, Andrew Dowsey (University of Bristol)

The University of Bristol Veterinary School has received a £1 million award from John Oldacre Foundation dedicated to creating an intensively monitored dairy farm at the school’s commercial housed facility, as a result, the John Oldacre Centre for Dairy Welfare & Sustainability Research was established. There have been many improvements implemented so far such as calf behaviour research areas, climate monitoring over a wireless network and over 60 cameras have been fitted for AI development. Further research infrastructure will include a flexible research pen for behavioural studies, and GreenFeed enteric methane emission monitoring equipment. The data collected at the farm from new equipment and sensors will be combined with data longitudinally collected at commercial farms such as individual production and reproductive data, genetics and veterinary records to answer fundamental challenges in livestock production related to health, welfare and sustainability. The farm manages an all-year-round calving herd made up of 200 milking cows split into two smaller groups (≤2 lactation and ≥3 lactation). The newly fitted video monitoring system covers all cattle housing areas (excluding the dry cow and heifer barns currently) including the current herringbone milking parlour as well as the entrance and exit to the parlour. The AI development for automated behaviour tracking of each individual is based on deep learning using what we refer to as open-set identification (Andrew et al., 2021). The AI system takes advantage of the unique dorsal coat patterns of Holstein-Friesian cows (black and white markings) for the identification of each individual. Through this novel approach the AI will not only detect, identify and track already known individuals but be able to recognise cattle it has not seen previously, removing the need to train the system for each new individual.

The specific management of the herd provides opportunities to continuously monitor cattle at different stages of the production cycle and at different parities. For example, monitoring changes in behaviour during the transition period, which is one of the most critical stages of production (Wankhade et al., 2017). The transition cow pen is monitored using six wide-angle cameras, enabling tracking of individuals across different views and monitoring behaviour such as social interactions, feeding, lying and drinking. The additional benefits of the transition pen are the smaller number of individuals present in camera view at one time as well as regular group composition changes that make this the optimal area for piloting work. Moreover, the exit from the milking parlour is of particular interest for the research surrounding body condition scoring (BCS) and mobility assessment of the herd. Automated AI-based BCS and mobility assessment methods will be trained using expert annotations of the side-view and back-view cameras. There is also a further camera pointed at the ear tag reader, as the cows exit the race, which uses optical character recognition to link the AI data with farm veterinary and production records based on the ear tag number.

Continuous monitoring of cattle using machine vision has the potential to characterise behaviour more accurately than current technologies used on farm (e.g. sensors/wearables), through off-cow sensing which avoids welfare concerns around wearable technologies (Tangorra et al., 2024). As such, it can be used to monitor more subtle changes in cattle behaviour for the detection of subclinical and clinical conditions in real-time. Behaviours such as feeding, lying and rumination have been reviewed previously and are known to change with incidence of disease (Dittrich, Gertz and Krieter, 2019). These behaviours can be referred to as “core behaviours” (Caplen and Held, 2021) which are critical to the immediate survival of the animal. Similarly, previous research has demonstrated changes in social behaviour such as displacements at the feed face (e.g. Sepulveda-Varas et al., 2016) and social reactivity (Caplen and Held, 2021), which can be referred to as “luxury behaviours” that have long-term, rather than immediate, health and welfare benefits (Caplen and Held,2021). Behaviours such as social interactions are believed to change prior to changes in maintenance behaviours based on sickness behaviour (Dittrich, Gertz and Krieter, 2019), and therefore present as an ideal candidate for early disease detection. Behaviour monitoring using visual recognition has been successfully applied previously, such as in wild Great Apes (Sakib and Burghardt 2020, Brookes et al., 2024) and is showing promise when applied to cattle social and individual behaviour. This research at University of Bristol will utilise our AI-infused annotation tool for behaviour scoring. This tool has a user-friendly interface to label BCS, mobility scores, as well as social interactions and individual behaviours to create a library of labelled data to be exploited for the development of automatic detection algorithms.

There are several challenges associated with the use of cameras to ensure the reliable detection of each individual cow which includes differentiating individuals in crowded areas (e.g. milking parlour entrance), re-identifying individuals between different camera views, and in diverse lighting. Additional work is being conducted to apply recognition algorithms across farms with various breeds that may not have distinctive coat patterns. This work focuses on depth images as a biometric measure for individual recognition (Sharma et al., 2025). The depth camera can also be used to optimize the BCS of cattle on farms, with the overall aim of improving the accuracy of the autonomous monitoring system.

Future work in this area of research at the John Oldacre Centre for Dairy Welfare & Sustainability Research will focus on collecting large datasets to optimize locations for capturing all behaviours of interest for automatic and disease detection. Further work will collect data at a number of representative farms across the UK and close collaboration with farmers and other stakeholders to translate the work for commercial use. This will include multiple workshops with the stakeholders to optimize the system and provide services to benefit farmers, such as, notifications of behavioural changes (otherwise called health alerts) as soon as they are detected. There are several advantages of an autonomous tracking system for veterinarians such as remote access to farm data and health metrics of individual animals and the herd as a whole. Despite these advantages it is key to keep the stakeholders at the heart of this development as these systems are aids to farm management not complete replacements of experienced staff. Therefore, the work being conducted at the John Oldacre Centre for Dairy Welfare & Sustainability Research in close collaboration with key stakeholders aims to deliver direction and an autonomous detection system for a more sustainable future in livestock production encompassing all aspects of sustainability.

Acknowledgements

Our current research is supported by BBSRC through grant BB/X017559/1 and the South West Biosciences Doctoral Training Partnership.

References

Andrew, W. et al. (2021) Visual identification of individual Holstein-Friesian cattle via deep metric learning. Computers and Electronics in Agriculture, 185, p.106133.

Brookes, O. et al. (2024) PanAf20K: A Large Video Dataset for Wild Ape Detection and Behaviour Recognition. International Journal of Computer Vision, 132(8), pp.3086-3102.

Caplen, G. and Held, S.D.E. (2021) Changes in social and feeding behaviors, activity, and salivary serum amyloid A in cows with subclinical mastitis. Journal of Dairy Science, 104(10), pp.10991-11008.

Dittrich, I., Gertz, M. and Krieter, J. (2019) Alterations in sick dairy cows’ daily behavioural patterns. Heliyon, 5(11), (Online) Available from: doi.org/10.1016/j.heliyon.2019.e02902 (Accessed 24/01/2024).

Tangorra, F.M. et al. (2024) Internet of Things (IoT): Sensors Application in Dairy Cattle Farming. Animals, 14(21), p.3071.

Sepúlveda-Varas, P., Proudfoot, K.L., Weary, D.M. and von Keyserlingk, M.A.G. Changes in behaviour of dairy cows with clinical mastitis. Appl Anim Behav Sci. 2016 Feb 1;175:8-13.

Sakib, F. and Burghardt, T. (2020). Visual Recognition of Great Ape Behaviours in the Wild arXiv:2011.10759.

Sharma, A. et al. (2025) Universal bovine identification via depth data and deep metric learning. Computers and Electronics in Agriculture, 229, p.109657.

Wankhade, P.R. et al. (2017) Metabolic and immunological changes in transition dairy cows: A review. Veterinary World, 10(11), pp.1367-1377.