arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.12481cs.CVcs.GR
收藏

NeuralFur: 从多视角图像中重建动物毛发

NeuralFur: Animal Fur Reconstruction From Multi-View Images

Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);ETH Zürich(苏黎世联邦理工学院);Technical University of Darmstadt(德累斯顿技术大学);University of Tübingen(图宾根大学)

Vanessa Sklyarova, Berna Kabadayi, Anastasios Yiannakidis, Giorgio Becherini, Michael J. Black, Justus Thies

AI总结:

NeuralFur通过视觉语言模型指导多视角图像重建,实现不同动物的高保真3D毛发建模。

摘要:

从图像中重建逼真的动物毛发几何结构是一项具有挑战性的任务,由于毛发的细尺度细节、自我遮挡以及视点依赖性外观。与人类发型重建不同,目前也没有可用于学习不同动物毛发先验知识的数据集。在本工作中,我们提出了一种基于多视角的高保真3D动物毛发建模方法,使用基于丝束的表示方法,并利用视觉语言模型的一般知识。给定多视角RGB图像,我们首先使用传统多视角立体技术重建粗略的表面几何结构。然后,我们使用视觉语言模型(VLM)系统检索每种身体部位的毛发真实长度结构信息。我们利用这一知识来构建动物的无毛几何结构,并在其上生长丝束。毛发重建通过多视角图像计算的几何和光度损失进行监督。为了缓解由于应用于输入图像的Gabor滤波器引起的取向歧义,我们还利用VLM来指导丝束的生长方向及其与我们纳入损失中的重力矢量的关系。通过使用VLM指导多视角输入的3D重建的新方案,我们展示了在不同毛发类型的动物上的一般化能力。对于额外结果和代码,请参考https://neuralfur.is.tue.mpg.de。

英文摘要:

Reconstructing realistic animal fur geometry from images is a challenging task due to the fine-scale details, self-occlusion, and view-dependent appearance of fur. In contrast to human hairstyle reconstruction, there are also no datasets that can be leveraged to learn a fur prior for different animals. In this work, we present a first multi-view-based method for high-fidelity 3D fur modeling of animals using a strand-based representation, leveraging the general knowledge of a vision language model. Given multi-view RGB images, we first reconstruct a coarse surface geometry using traditional multi-view stereo techniques. We then use a vision language model (VLM) system to retrieve information about the realistic length structure of the fur for each part of the body. We use this knowledge to construct the animal's furless geometry and grow strands atop it. The fur reconstruction is supervised with both geometric and photometric losses computed from multi-view images. To mitigate orientation ambiguities stemming from the Gabor filters that are applied to the input images, we additionally utilize the VLM to guide the strands' growth direction and their relation to the gravity vector that we incorporate as a loss. With this new schema of using a VLM to guide 3D reconstruction from multi-view inputs, we show generalization across a variety of animals with different fur types. For additional results and code, please refer to https://neuralfur.is.tue.mpg.de.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑