DiffCAD：从RGB图像进行弱监督概率性CAD模型检索与对齐
DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Image
AI总结：
DiffCAD提出首个弱监督概率性CAD检索与对齐方法，利用扩散模型从单张RGB图像生成多假设CAD重建，仅用合成数据训练即可在Scan2CAD上超越有监督SOTA 5.9%。
摘要：
基于CAD模型基元从RGB图像中感知三维结构，可以实现有效且高效的基于三维物体的场景表示。然而，当前方法依赖与真实图像相关联的CAD模型昂贵标注作为监督，并且由于任务中固有的歧义性而面临挑战——既包括单目感知中的深度尺度歧义，也包括CAD数据库模型与真实观测之间的不精确匹配。因此，我们提出DiffCAD，这是首个从RGB图像进行CAD检索与对齐的弱监督概率方法。我们将该任务形式化为条件生成任务，利用扩散模型学习隐式概率模型，以捕捉图像中CAD物体的形状、姿态和尺度。这使得能够对多种合理的CAD重建进行多假设生成，仅需少量假设即可刻画深度/尺度以及不精确形状匹配中的歧义性。我们的方法仅在合成数据上训练，利用单目深度和掩码估计实现对各种真实目标域的鲁棒零样本适应。尽管仅在合成数据上训练，我们的多假设方法在Scan2CAD数据集上使用8个假设时，甚至可以超越有监督的最先进方法5.9%。
英文摘要：
Perceiving 3D structures from RGB images based on CAD model primitives can enable an effective, efficient 3D object-based representation of scenes. However, current approaches rely on supervision from expensive annotations of CAD models associated with real images, and encounter challenges due to the inherent ambiguities in the task -- both in depth-scale ambiguity in monocular perception, as well as inexact matches of CAD database models to real observations. We thus propose DiffCAD, the first weakly-supervised probabilistic approach to CAD retrieval and alignment from an RGB image. We formulate this as a conditional generative task, leveraging diffusion to learn implicit probabilistic models capturing the shape, pose, and scale of CAD objects in an image. This enables multi-hypothesis generation of different plausible CAD reconstructions, requiring only a few hypotheses to characterize ambiguities in depth/scale and inexact shape matches. Our approach is trained only on synthetic data, leveraging monocular depth and mask estimates to enable robust zero-shot adaptation to various real target domains. Despite being trained solely on synthetic data, our multi-hypothesis approach can even surpass the supervised state-of-the-art on the Scan2CAD dataset by 5.9% with 8 hypotheses.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
