NeuralFur: Animal Fur Reconstruction From Multi-View Images
Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);ETH Zürich(苏黎世联邦理工学院);Technical University of Darmstadt(德累斯顿技术大学);University of Tübingen(图宾根大学)
Vanessa Sklyarova, Berna Kabadayi, Anastasios Yiannakidis, Giorgio Becherini, Michael J. Black, Justus Thies
Reconstructing realistic animal fur geometry from images is a challenging task due to the fine-scale details, self-occlusion, and view-dependent appearance of fur. In contrast to human hairstyle reconstruction, there are also no datasets that can be leveraged to learn a fur prior for different animals. In this work, we present a first multi-view-based method for high-fidelity 3D fur modeling of animals using a strand-based representation, leveraging the general knowledge of a vision language model. Given multi-view RGB images, we first reconstruct a coarse surface geometry using traditional multi-view stereo techniques. We then use a vision language model (VLM) system to retrieve information about the realistic length structure of the fur for each part of the body. We use this knowledge to construct the animal's furless geometry and grow strands atop it. The fur reconstruction is supervised with both geometric and photometric losses computed from multi-view images. To mitigate orientation ambiguities stemming from the Gabor filters that are applied to the input images, we additionally utilize the VLM to guide the strands' growth direction and their relation to the gravity vector that we incorporate as a loss. With this new schema of using a VLM to guide 3D reconstruction from multi-view inputs, we show generalization across a variety of animals with different fur types. For additional results and code, please refer to https://neuralfur.is.tue.mpg.de.
MoZoo:Unleashing Video Diffusion power in animal fur and muscle simulation
tabular c 1 Tsinghua University 2 University of Glasgow 3 The Chinese University of Hong Kong 4 HUIJING Digital Media \& Entertainment Group 2mm Project Page: tabular
AnimalTrack: A Benchmark for Multi-Animal Tracking in the Wild
Institute of Software Chinese Academy of Sciences(中国科学院软件研究所);University of Chinese Academy of Sciences(中国科学院大学);Nanjing Institute of Software Technology(南京软件技术研究院);University of North Texas(北德克萨斯大学)
SAM 3D Animal: Promptable Animal 3D Reconstruction from Images in the Wild
University of Cambridge(剑桥大学);Southern University of Science and Technology(南方科技大学);Tsinghua University(清华大学);IMATI-CNR, Milan, Italy(意大利米兰IMATI-CNR研究所)
本文提出SAM 3D Animal,首个可提示的多动物3D重建框架,基于SMAL+参数化动物模型,通过关键点和掩码提示实现拥挤和遮挡场景的可靠重建,利用Herd3D数据集在多个数据集上取得最优结果。
Unsupervised Pelage Pattern Unwrapping for Animal Re-identification
Computer Vision and Pattern Recognition Laboratory (CVPRL)(计算机视觉与模式识别实验室);Lappeenranta-Lahti University of Technology LUT(拉佩恩拉-拉赫蒂理工大学);Faculty of Information Technology Brno University of Technology (BUT)(信息科技学院布拉格技术大学);Rensselaer Polytechnic Institute (RPI)(伦塞拉尔理工学院)
VastMAT: A Large-Scale Multi-Category Benchmark for Multi-Animal Tracking
Dalian University of Technology(大连理工大学);Wuhan University(武汉大学);University of North Texas(北德克萨斯大学);The Hong Kong University of Science and Technology(香港科技大学)
Vision Transformers for Zero-Shot Clustering of Animal Images: A Comparative Benchmarking Study
Department of Chemistry and Bioscience(化学与生物科学系);Aalborg University(奥尔堡大学);Department of Architecture, Design and Media Technology(建筑、设计与媒体技术系);Visual Analysis and Perception Lab(视觉分析与感知实验室)
3D-MuPPET: 3D Multi-Pigeon Pose Estimation and Tracking
University of Konstanz(康斯坦茨大学);Max Planck Institute of Animal Behavior(马克斯·普朗克动物行为研究所);Eötvös Loránd University(厄特沃什·罗兰大学);Hungarian Academy of Sciences(匈牙利科学院)
Application of deep learning to camera trap data for ecologists in planning / engineering -- Can captivity imagery train a model which generalises to the wild?
Ove Arup and Partners(奥雅纳工程顾问公司);Newcastle University(纽卡斯尔大学)
了解一个物种的丰富程度是了解其长期可持续性和我们可能对其产生的影响的第一步。生态学家使用相机陷阱远程调查特定动物物种的存在。先前的研究表明,深度学习模型可以被训练成自动检测和分类相机陷阱图像中的动物,具有很高的置信度。然而,训练这些模型的能力依赖于具有足够高质量的训练数据。当动物很稀有或者数据集不存在时会发生什么?本研究提出了一种方法,使用圈养的稀有动物(重点是苏格兰野猫)的图像来生成训练数据集。我们探讨了将人工饲养数据训练的模型应用于野外收集的数据时所面临的挑战。该研究的背景是生态学家在规划/工程的需求。根据其他研究的先例,该项目建立了一个合奏目标检测,图像分割和图像分类模型,然后使用不同的图像处理和类结构技术进行测试,以鼓励模型的推广。该研究的结论是,在苏格兰野猫的背景下,圈养图像训练的模型不能推广到野生相机陷阱图像使用现有的技术。然而,基于两类模型Wildcat vs Not Wildcat的最终模型性能在只有1%的图像包含野猫的测试集上实现了81.6%的总体准确性得分和54.8%的野猫准确性得分。这表明使用圈养图像是可行的,需要进一步研究。这是第一个研究,试图产生一个训练集的基础上圈养的数据和第一个探索这种模式的发展环境中的生态学家在规划/工程。