与邻居共享:通过跨实例一致性进行单视图重建
Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency
摘要:
单视图重建的方法通常依赖于视点注释、轮廓、背景的缺失、同一实例的多个视图、模板形状或对称性。我们通过明确地利用不同对象实例的图像之间的一致性来避免所有这些监督和假设。因此,我们的方法可以从大量描绘相同对象类别的未标记图像中学习。我们的主要贡献是利用跨实例一致性的两种方法:(i)渐进式条件反射,一种训练策略,以课程学习的方式逐渐将模型从类别专业化到实例;(ii)邻居重建,具有相似形状或纹理的实例之间的一致性损失。我们的方法的成功也至关重要:我们的结构化自动编码架构将图像分解为明确的形状,纹理,姿势和背景;差分渲染的适应性配方;以及3D和姿势学习之间交替的新优化方案。我们比较了我们的方法,UNICORN,在不同的合成ShapeNet数据集上-需要多个视图作为监督的方法的经典基准-以及标准的真实图像基准(Pascal 3D + Car,CUB),其中大多数方法需要已知的模板和轮廓注释。我们还展示了适用于更具挑战性的现实世界的集合(CompCars,LSUN),其中轮廓不可用,图像不会在对象周围裁剪。
英文摘要:
Approaches for single-view reconstruction typically rely on viewpoint annotations, silhouettes, the absence of background, multiple views of the same instance, a template shape, or symmetry. We avoid all such supervision and assumptions by explicitly leveraging the consistency between images of different object instances. As a result, our method can learn from large collections of unlabelled images depicting the same object category. Our main contributions are two ways for leveraging cross-instance consistency: (i) progressive conditioning, a training strategy to gradually specialize the model from category to instances in a curriculum learning fashion; and (ii) neighbor reconstruction, a loss enforcing consistency between instances having similar shape or texture. Also critical to the success of our method are: our structured autoencoding architecture decomposing an image into explicit shape, texture, pose, and background; an adapted formulation of differential rendering; and a new optimization scheme alternating between 3D and pose learning. We compare our approach, UNICORN, both on the diverse synthetic ShapeNet dataset - the classical benchmark for methods requiring multiple views as supervision - and on standard real-image benchmarks (Pascal3D+ Car, CUB) for which most methods require known templates and silhouette annotations. We also showcase applicability to more challenging real-world collections (CompCars, LSUN), where silhouettes are not available and images are not cropped around the object.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。