arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.09203cs.CV
收藏

Cattle-CLIP:一种用于从视频中识别牛行为的多模态框架

Cattle-CLIP: A Multimodal Framework for Dairy Cattle Behaviour Recognition from Video

Bristol Veterinary School, University of Bristol(布里斯托尔大学布里斯托尔兽医学院);School of Computer Science, University of Bristol(布里斯托尔大学计算机科学学院);Joint Institute of Qingdao Huanghai University and WeITec, Qingdao Huanghai University(青岛黄海学院与WeITec联合学院,青岛黄海学院)

Huimin Liu, Jing Gao, Daria Baran, Axel X Montout, Neill W Campbell, Andrew W Dowsey

AI总结:

本文提出Cattle-CLIP框架,通过跨模态语义对齐解决农场环境中的行为识别难题,构建了CattleBehaviors6数据集,实现了高准确率的行为识别。

摘要:

现实农场环境中鲁棒的行为识别仍然具有挑战性,由于数据相关限制,包括良好标注的牲畜视频数据集稀缺以及大规模预训练语料与农业监控视频之间的显著领域差距。为了解决这些挑战,我们提出了Cattle-CLIP,一种领域自适应的视觉-语言框架,将牛行为识别重新表述为跨模态语义对齐,而非纯粹的视觉分类。Cattle-CLIP不直接微调视觉主干,而是结合时间整合模块,将图像级对比预训练扩展到基于视频的行为理解,实现时间上的语义对齐。为了缓解预训练模型所用的网络级图像-文本数据与现实世界牛监控视频之间的分布偏移,我们进一步引入了定制化的增强策略和专门的行为提示。此外,我们构建了CattleBehaviors6,一个经过精心挑选且行为一致的视频数据集,包含1905个标注片段,涵盖六种室内行为,以支持模型训练和评估。除了作为我们所提方法的基准外,该数据集还提供了一种标准化的乙状带定义,为未来牲畜行为分析研究提供了实用资源。Cattle-CLIP在完全监督和少样本学习场景下进行评估,特别关注数据稀缺的行为识别,这是牲畜监控中一个重要但尚不充分研究的目标。实验表明,Cattle-CLIP在监督设置下实现了六个行为的96.1%整体准确率,对进食、饮水和站立反刍行为的召回率接近完美,并在少样本场景下展示了鲁棒的泛化能力。

英文摘要:

Monitoring cattle behaviour provides valuable insights into animal health, productivity and welfare. However, robust behaviour recognition in real-world farm environments remains challenging because well-annotated livestock video datasets are scarce and agricultural surveillance footage differs substantially from large-scale pretraining data. To address these challenges, we propose Cattle-CLIP, a domain-adaptive vision-language framework based on Contrastive Language-Image Pretraining (CLIP), which reformulates cattle behaviour recognition as a cross-modal semantic alignment task. Cattle-CLIP incorporates a temporal integration module that extends image-level contrastive pretraining to video-based behaviour understanding and promotes consistent semantic alignment across time. To reduce domain discrepancy, customised data augmentation and behaviour-oriented prompts are employed. We also construct CattleBehaviours6, a curated dataset containing 1,905 annotated clips from six indoor behaviours: feeding, drinking, standing-self-grooming, standing-ruminating, lying-self-grooming and lying-ruminating. The dataset provides standardised ethogram definitions for model training, evaluation and future livestock behaviour research. Cattle-CLIP was evaluated under supervised and few-shot learning settings, with particular emphasis on data-constrained behaviour recognition. In supervised experiments, the model achieved 96.1% accuracy across six behaviours and showed high precision and recall for feeding, drinking and standing-ruminating. It also maintained promising few-shot performance with limited labelled data, demonstrating the potential of multimodal learning for cattle behaviour recognition in data-constrained farm environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑