BARC:通过利用品种信息从图像学习回归3D狗形
BARC: Learning to Regress 3D Dog Shape from Images by Exploiting Breed Information
摘要:
我们的目标是从单个图像中恢复狗的3D形状和姿势。这是一项具有挑战性的任务,因为狗表现出各种各样的形状和外观,并且高度清晰。最近的工作提出了直接回归SMAL动物模型,与其他肢体尺度参数,从图像。我们的方法称为BARC(使用分类的品种增强回归),在几个重要方面超越了以前的工作。首先,我们修改SMAL形状空间,使其更适合于表示狗的形状。但是,即使有更好的形状模型,从图像中回归狗形状的问题仍然具有挑战性,因为我们缺乏具有3D地面实况的配对图像。为了弥补配对数据的缺乏,我们制定了新的损失,利用狗品种的信息。特别是,我们利用了同一品种的狗有相似的身体形状的事实。我们制定了一个新的品种相似性损失包括两个部分:一个术语鼓励来自同一品种的狗的形状比不同品种的狗更相似。第二个,品种分类损失,有助于产生可识别的品种特定的形状。通过消融研究,我们发现,我们的品种损失显着提高形状精度超过基线没有他们。我们还比较BARC定性与WLDO的知觉研究,发现我们的方法产生的狗是显着更现实。这项工作表明,遗传相似性的先验信息可以帮助弥补缺乏三维训练数据。这一概念可能适用于其他动物物种或物种群。我们的代码可在https://barc.is.tue.mpg.de/上公开获取以供研究。
英文摘要:
Our goal is to recover the 3D shape and pose of dogs from a single image. This is a challenging task because dogs exhibit a wide range of shapes and appearances, and are highly articulated. Recent work has proposed to directly regress the SMAL animal model, with additional limb scale parameters, from images. Our method, called BARC (Breed-Augmented Regression using Classification), goes beyond prior work in several important ways. First, we modify the SMAL shape space to be more appropriate for representing dog shape. But, even with a better shape model, the problem of regressing dog shape from an image is still challenging because we lack paired images with 3D ground truth. To compensate for the lack of paired data, we formulate novel losses that exploit information about dog breeds. In particular, we exploit the fact that dogs of the same breed have similar body shapes. We formulate a novel breed similarity loss consisting of two parts: One term encourages the shape of dogs from the same breed to be more similar than dogs of different breeds. The second one, a breed classification loss, helps to produce recognizable breed-specific shapes. Through ablation studies, we find that our breed losses significantly improve shape accuracy over a baseline without them. We also compare BARC qualitatively to WLDO with a perceptual study and find that our approach produces dogs that are significantly more realistic. This work shows that a-priori information about genetic similarity can help to compensate for the lack of 3D training data. This concept may be applicable to other animal species or groups of species. Our code is publicly available for research purposes at https://barc.is.tue.mpg.de/.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。