We introduce a multimodal vision framework for precision livestock farming, harnessing the power of GroundingDINO, HQSAM, and ViTPose models. This integrated suite enables comprehensive behavioral analytics from video data without invasive animal tagging. GroundingDINO generates accurate bounding boxes around livestock, while HQSAM segments individual animals within these boxes. ViTPose estimates key body points, facilitating posture and movement analysis. Demonstrated on a sheep dataset with grazing, running, sitting, standing, and walking activities, our framework extracts invaluable insights: activity and grazing patterns, interaction dynamics, and detailed postural evaluations. Applicable across species and video resolutions, this framework revolutionizes non-invasive livestock monitoring for activity detection, counting, health assessments, and posture analyses. It empowers data-driven farm management, optimizing animal welfare and productivity through AI-powered behavioral understanding.
A physics-informed, behavior-aware digital twin for robust multimodal forecasting of core body temperature in precision livestock farming
Applied Artificial Intelligence and Intelligent Systems (AAIINS) Laboratory(应用人工智能与智能系统实验室);Department of Computer Science and Engineering(计算机科学与工程系);Department of Data Science and Artificial Intelligence(数据科学与人工智能系);Department of Software Systems & Cybersecurity(软件系统与网络安全系);Energy and Resources Institute, Faculty of Science and Technology(能源与资源研究所,科学与技术学院);Faculty of Science and Technology(科学与技术学院);School of Engineering and Energy(工程与能源学院)
A Multicore and Edge TPU-Accelerated Multimodal TinyML System for Livestock Behavior Recognition
Medical AI Lab, Hebei Provincial Engineering Research Center for AI-Based Cancer Treatment Decision-Making, The First Hospital of Hebei Medical University(医学人工智能实验室,河北省人工智能辅助癌症治疗决策工程研究中心,河北省医科大学第一医院);Computing Department, Imperial College London(计算部门,帝国理工学院伦敦分校);Professor Pervasive Sensing & TinyML and the Head of the Smart Sensing Lab at Nottingham Trent University(感知与TinyML教授及智能感知实验室主任,诺丁汉特伦特大学);Provost’s Visiting Professor in tinyML at Imperial College London(帝国理工学院伦敦分校副校长兼任TinyML客座教授)
GPT-4o: Visual perception performance of multimodal large language models in piglet activity understanding
China Agricultural University(中国农业大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);AniEye Technology(安尼眼科技);State Key Laboratory of Animal Nutrition and Feeding(动物营养与饲料国家重点实验室)
Technical note: ShinyAnimalCV: open-source cloud-based web application for object detection, segmentation, and three-dimensional visualization of animals using computer vision
University of Florida(佛罗里达大学);North Carolina State University(北卡罗来纳州立大学);Virginia Polytechnic Institute and State University(弗吉尼亚理工学院暨州立大学)
Cattle-CLIP: A Multimodal Framework for Dairy Cattle Behaviour Recognition from Video
Bristol Veterinary School, University of Bristol(布里斯托尔大学布里斯托尔兽医学院);School of Computer Science, University of Bristol(布里斯托尔大学计算机科学学院);Joint Institute of Qingdao Huanghai University and WeITec, Qingdao Huanghai University(青岛黄海学院与WeITec联合学院,青岛黄海学院)
Exploring parameter-efficient fine-tuning (PEFT) of billion-parameter vision models with QLoRA and DoRA: insights into generalization for limited-data image classification under a 98:1 test-to-train regime
Cornell University, College of Agriculture and Life Sciences(康奈尔大学农业与生命科学学院)
Do Multimodal Large Language Models Understand Welding?
University of Notre Dame(圣母大学);Keough School of Global Affairs, University of Notre Dame(圣母大学基全球事务学院);Industry Labs, University of Notre Dame(圣母大学产业实验室)
BioAutoML-NAS: An End-to-End AutoML Framework for Multimodal Insect Classification via Neural Architecture Search on Large-Scale Biodiversity Data
Department of Computer Science and Engineering, United International University, Dhaka, Bangladesh(乌姆国际大学计算机科学与工程系,达卡,孟加拉国);Applied Artificial Intelligence and Intelligent Systems (AAIINS) Laboratory, 1217, Dhaka, Bangladesh(应用人工智能与智能系统实验室(AAIINS),1217号,达卡,孟加拉国);Department of Computer Science and Engineering, Penn State University, University Park, PA, USA(宾夕法尼亚州立大学计算机科学与工程系,University Park,PA,美国);Faculty of Science and Information Technology, Charles Darwin University, Sydney, NSW, Australia(查尔斯达尔文大学科学与信息技术学院,悉尼,新南威尔士州,澳大利亚);Faculty of Science and Technology, Charles Darwin University, Casuarina, 0909, NT, Australia(查尔斯达尔文大学科学与技术学院,Casuarina,0909,北领地,澳大利亚)
Direct Video-Based Spatiotemporal Deep Learning for Cattle Lameness Detection
Department of Software Engineering(软件工程系);Daffodil International University(达福尔国际大学);Department of Computer Science, Faculty of Computer Science and Information Technology(计算机科学系);King Faisal University(国王法赫德大学)