LUDVIG：将2D视觉特征免学习提升至Gaussian Splatting场景
LUDVIG: Learning-Free Uplifting of 2D Visual Features to Gaussian Splatting Scenes
AI总结：
提出免学习方法LUDVIG，通过特征聚合与图扩散将2D视觉基础模型特征提升至Gaussian Splatting三维场景，在分割等任务上达到可比性能并显著提速。
摘要：
我们致力于解决将DINO、SAM和CLIP等视觉基础模型的能力扩展到3D任务的问题。具体而言，我们提出了一种将2D图像特征提升为3D场景的Gaussian Splatting表示的新方法。与依赖最小化重建损失的传统方法不同，我们的方法采用了一种更简单、更高效的特征聚合技术，并辅以图扩散机制。图扩散利用3D几何结构和DINOv2诱导的成对相似性来细化3D特征，例如粗糙分割掩码。我们的方法在多个下游任务上取得了与最先进技术相当的性能，同时实现了显著的加速。值得注意的是，尽管DINOv2没有像SAM那样在数百万个带标注的分割掩码上进行训练，我们仅使用通用DINOv2特征就获得了具有竞争力的分割结果。当应用于CLIP特征时，我们的方法在开放词汇对象分割任务中表现出强大性能，凸显了该方法的多功能性。
英文摘要：
We address the problem of extending the capabilities of vision foundation models such as DINO, SAM, and CLIP, to 3D tasks. Specifically, we introduce a novel method to uplift 2D image features into Gaussian Splatting representations of 3D scenes. Unlike traditional approaches that rely on minimizing a reconstruction loss, our method employs a simpler and more efficient feature aggregation technique, augmented by a graph diffusion mechanism. Graph diffusion refines 3D features, such as coarse segmentation masks, by leveraging 3D geometry and pairwise similarities induced by DINOv2. Our approach achieves performance comparable to the state of the art on multiple downstream tasks while delivering significant speed-ups. Notably, we obtain competitive segmentation results using only generic DINOv2 features, despite DINOv2 not being trained on millions of annotated segmentation masks like SAM. When applied to CLIP features, our method demonstrates strong performance in open-vocabulary object segmentation tasks, highlighting the versatility of our approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
