图像-形状对应关系的最佳分割伙伴
Best Segmentation Buddies for Image-Shape Correspondence
AI总结：
本文研究了在真实图像和无纹理3D形状之间估计分割到分割对应关系的任务，通过将图像像素与3D形状顶点链接，利用深度视觉特征进行特征相似性计算，从而实现跨模态的对应关系发现。
摘要：
寻找对应关系是计算机视觉和图形学中的基本且广泛研究的问题。在本工作中，我们探讨了在真实图像和无纹理3D形状之间估计分割到分割对应关系的任务。该任务极具挑战性，因为存在显著的外观、几何和视角差异。我们的方法通过将图像分割中的像素链接到相应3D形状语义部分的顶点，跨越了跨模态的差距。为此，我们首先将深度视觉特征从2D视觉模型蒸馏到3D形状表面，使得能够计算图像像素与形状顶点之间的特征相似性。然后，我们识别出最佳分割伙伴，即那些最相似的图像像素位于图像分割区域内的顶点，从而可靠地发现语义对应形状部分的顶点。最后，我们利用从2D图像分割模型蒸馏的3D特征，直接在3D中对形状进行分割，从而引导对应关系的发现。我们展示了我们的方法在广泛图像-形状配对中的通用性和鲁棒性，展示了准确且具有语义意义的对应关系。我们的项目页面位于https://threedle.github.io/bsb/。
英文摘要：
Finding correspondences is a fundamental and extensively researched problem in computer vision and graphics. In this work, we examine the underexplored task of estimating segmentation-to-segmentation correspondence between images in the wild and untextured 3D shapes. This task is highly challenging due to substantial differences in appearance, geometry, and viewpoint. Our approach bridges the cross-modality gap by linking pixels in the image segment to vertices in the corresponding semantic part of the 3D shape. To achieve this, we first distill deep visual features from a 2D vision model onto the 3D shape surface, allowing for the computation of feature similarity between image pixels and shape vertices. Then, we identify Best Segmentation Buddies, vertices whose most similar image pixel lies within the image segmentation region, enabling the reliable discovery of vertices in semantically corresponding shape parts. Finally, we leverage distilled 3D features from the 2D image segmentation model to segment the shape directly in 3D, bootstrapping the correspondence process. We demonstrate the generality and robustness of our approach across a wide range of image-shape pairs, showcasing accurate and semantically meaningful correspondences. Our project page is at https://threedle.github.io/bsb/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
