OpenObj：具有细粒度理解能力的开放词汇对象级神经辐射场
OpenObj: Open-Vocabulary Object-Level Neural Radiance Fields with Fine-Grained Understanding
AI总结：
本文提出 OpenObj 框架，通过融合对象级实例与部件级特征构建开放词汇 NeRF，在零样本分割、检索及多尺度机器人任务中表现优异。
摘要：
近年来，在视觉语言模型（VLM）的推动下，开放词汇三维场景重建受到广泛关注，这些模型在开集检索方面展现出卓越能力。然而，现有方法存在一些局限：它们要么专注于学习逐点特征，导致语义理解模糊；要么仅处理对象级重建，从而忽略对象内部的复杂细节。为应对这些挑战，我们提出 OpenObj，这是一种构建具有细粒度理解能力的开放词汇对象级神经辐射场（NeRF）的创新方法。本质上，OpenObj 建立了一个稳健框架，可在对象级实现高效且水密的场景建模与理解。此外，我们将部件级特征融入神经场，从而能够细致表示对象内部。该方法在保持细粒度理解的同时捕获对象级实例。在多个数据集上的结果表明，OpenObj 在零样本语义分割和检索任务中取得了更优性能。此外，OpenObj 还支持多种尺度的真实机器人任务，包括全局移动和局部操作。
英文摘要：
In recent years, there has been a surge of interest in open-vocabulary 3D scene reconstruction facilitated by visual language models (VLMs), which showcase remarkable capabilities in open-set retrieval. However, existing methods face some limitations: they either focus on learning point-wise features, resulting in blurry semantic understanding, or solely tackle object-level reconstruction, thereby overlooking the intricate details of the object's interior. To address these challenges, we introduce OpenObj, an innovative approach to build open-vocabulary object-level Neural Radiance Fields (NeRF) with fine-grained understanding. In essence, OpenObj establishes a robust framework for efficient and watertight scene modeling and comprehension at the object-level. Moreover, we incorporate part-level features into the neural fields, enabling a nuanced representation of object interiors. This approach captures object-level instances while maintaining a fine-grained understanding. The results on multiple datasets demonstrate that OpenObj achieves superior performance in zero-shot semantic segmentation and retrieval tasks. Additionally, OpenObj supports real-world robotics tasks at multiple scales, including global movement and local manipulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
