学习网络上的三维动物群
Learning the 3D Fauna of the Web
AI总结:
本文提出3D-Fauna方法,利用互联网二维图像和蒙皮模型语义库自动发现基础形状,联合学习100多种动物的可变形三维模型,并构建大规模数据集,实现从单张四足动物图像快速重建关节化三维网格。
摘要:
学习地球上所有动物的三维模型需要大规模扩展现有解决方案。怀着这一终极目标,我们开发了3D-Fauna,这是一种联合学习超过100种动物物种的泛类别可变形三维动物模型的方法。动物建模的一个关键瓶颈是训练数据的可用性有限,我们通过简单地从二维互联网图像中学习来克服这一问题。我们表明,先前的类别特定方法无法泛化到训练图像有限的稀有物种。我们通过引入蒙皮模型语义库(SBSM)来应对这一挑战,该语义库通过将几何归纳先验与现成的自监督特征提取器隐式捕获的语义知识相结合,自动发现一小组基础动物形状。为了训练这样的模型,我们还贡献了一个新的大规模多样化动物物种数据集。在推理时,给定任何四足动物的单张图像,我们的模型能在几秒钟内以前馈方式重建出关节化的三维网格。
英文摘要:
Learning 3D models of all animals on the Earth requires massively scaling up existing solutions. With this ultimate goal in mind, we develop 3D-Fauna, an approach that learns a pan-category deformable 3D animal model for more than 100 animal species jointly. One crucial bottleneck of modeling animals is the limited availability of training data, which we overcome by simply learning from 2D Internet images. We show that prior category-specific attempts fail to generalize to rare species with limited training images. We address this challenge by introducing the Semantic Bank of Skinned Models (SBSM), which automatically discovers a small set of base animal shapes by combining geometric inductive priors with semantic knowledge implicitly captured by an off-the-shelf self-supervised feature extractor. To train such a model, we also contribute a new large-scale dataset of diverse animal species. At inference time, given a single image of any quadruped animal, our model reconstructs an articulated 3D mesh in a feed-forward fashion within seconds.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。