OpenScan: 一个用于通用开放词汇3D场景理解的基准
OpenScan: A Benchmark for Generalized Open-Vocabulary 3D Scene Understanding
AI总结：
OpenScan基准旨在评估模型对3D场景的通用开放词汇理解能力，发现现有方法难以处理抽象词汇，需探索新方向。
摘要：
开放词汇3D场景理解（OV-3D）旨在定位和分类超出封闭对象类别的新对象。然而，现有方法和基准主要集中在对象类别的开放词汇问题上，这在全面评估模型对3D场景理解程度方面是不足的。在本文中，我们引入了一个更具挑战性的任务，称为通用开放词汇3D场景理解（GOV-3D），以探索超越对象类别的开放词汇问题。它涵盖了一个开放且多样的通用知识集，以细粒度和对象特定属性的语义查询形式表达。为此，我们贡献了一个新的基准，名为OpenScan，其中包括八个代表性语言方面的3D对象属性，包括功能、属性和材料。我们进一步在我们的OpenScan基准上评估了最先进的OV-3D方法，并发现这些方法难以理解GOV-3D任务的抽象词汇，这一挑战无法通过在训练期间扩大对象类别来简单解决。我们强调了现有方法的局限性，并探索了克服已识别不足的有希望的方向。
英文摘要：
Open-vocabulary 3D scene understanding (OV-3D) aims to localize and classify novel objects beyond the closed set of object classes. However, existing approaches and benchmarks primarily focus on the open vocabulary problem within the context of object classes, which is insufficient in providing a holistic evaluation to what extent a model understands the 3D scene. In this paper, we introduce a more challenging task called Generalized Open-Vocabulary 3D Scene Understanding (GOV-3D) to explore the open vocabulary problem beyond object classes. It encompasses an open and diverse set of generalized knowledge, expressed as linguistic queries of fine-grained and object-specific attributes. To this end, we contribute a new benchmark named \textit{OpenScan}, which consists of 3D object attributes across eight representative linguistic aspects, including affordance, property, and material. We further evaluate state-of-the-art OV-3D methods on our OpenScan benchmark and discover that these methods struggle to comprehend the abstract vocabularies of the GOV-3D task, a challenge that cannot be addressed simply by scaling up object classes during training. We highlight the limitations of existing methodologies and explore promising directions to overcome the identified shortcomings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
