面向3D视觉定位的多样化语言生成扩展
Scaling Diverse Language Generation for 3D Visual Grounding
AI总结：
提出ViGiL3D++方法，通过场景图约束采样与LLM语言生成结合，生成多样化视觉定位查询，提升3DVG模型泛化能力并揭示VLM局限性。
摘要：
开发用于3D视觉定位（3DVG）的鲁棒模型——即在自然语言描述的3D场景中定位实体——对于使智能体能够将空间语言与物理世界中的对象对应起来至关重要。然而，缺乏大规模多样化的描述阻碍了模型泛化到简单语言模式之外。最近的此类尝试在用于定位对象的约束类型和语言上缺乏多样性。描述方法无法精确对比对象，而这对于视觉定位很重要。因此，我们提出ViGiL3D++，一种可扩展的、场景无关的方法，通过将场景图中的约束采样与LLM的语言生成相结合，生成多样化的视觉定位查询。我们表明，与现有的扩展数据集相比，它具有更大的多样性，并在多个3DVG基准测试上提高了模型性能，同时也揭示了VLM的突出局限性。
英文摘要：
Developing robust models for 3D visual grounding (3DVG), the localization of entities in a 3D scene described in natural language, is important for enabling agents to correspond spatial language with objects in the physical world. However, the lack of diverse descriptions at scale prevents models from generalizing beyond simple linguistic patterns. Recent such attempts lack diversity in the constraint types and language used to ground objects. Captioning methods cannot precisely contrast objects, which is important for visual grounding. We therefore propose ViGiL3D++, a scalable, scene-agnostic method that generates diverse visual grounding queries by combining constraint sampling in scene graphs with the language generation of LLMs. We show that it has greater diversity over existing scaled datasets and improves model performance over several 3DVG benchmarks but also illuminates outstanding limitations of VLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
