深入场景：通过焦点计划生成打破视觉-语言决策中的感知瓶颈
Dive into the Scene: Breaking the Perceptual Bottleneck in Vision-Language Decision Making via Focus Plan Generation
AI总结：
提出SceneDiver方法，通过从粗到细的焦点计划生成，逐步构建场景图并分解任务，减少视觉幻觉，提升视觉-语言模型和视觉-语言-动作模型在具身决策任务中的表现。
摘要：
在具身视觉-语言决策任务（如机器人操作和导航）中，视觉-语言模型和视觉-语言-动作模型（VLMs & VLAs）是具有不同优势的强大工具：VLMs更擅长长期规划，而VLAs更擅长反应控制。然而，它们的性能受到相同感知瓶颈的限制：由于模型无法区分任务相关对象与干扰物，导致视觉幻觉。原则上，准确识别并聚焦关键对象同时过滤无关对象是突破这一限制的关键。一个直接的解决方案是一步聚焦：直接关注重要对象。然而，这种方法被证明无效，因为有效的聚焦本质上需要深度场景理解。为此，我们提出SceneDiver，一种利用VLMs长期规划能力的从粗到细的焦点计划生成方法，首先构建整体场景图以建立初步理解，然后通过识别、理解和分析的迭代循环逐步将任务分解为更简单的子问题。为了实现反应控制，我们还设计了一个轻量级适配器，将深思熟虑的聚焦能力蒸馏到VLAs中。在标准具身AI基准上的评估证实，我们的方法显著减少了VLMs和VLAs的视觉幻觉，同时在需要快速执行的任务中保持了计算效率。我们的代码和数据发布在：https://future-item.github.io/SceneDiver。
英文摘要：
In embodied vision-language decision making tasks such as robotic manipulation and navigation, Vision-Language and Vision-Language-Action Models (VLMs & VLAs) are powerful tools with different benefits: VLMs are better at long-term planning, while VLAs are better at reactive control. However, their performance is limited by the same perceptual bottleneck: visual hallucinations arise due to the models' inability to distinguish task-relevant objects from distractors. In principle, accurate identification and focus on critical objects while filtering out irrelevant ones is the key to break this limitation. A straightforward solution is one-step focus: directly attending to essential objects. However, this approach proves ineffective because effective focus inherently requires deep scene understanding. To this end, we propose SceneDiver, a coarse-to-fine focus plan generation method for VLMs leveraging their long-term planning abilities, that first constructs a holistic scene graph to establish initial comprehension, then progressively decomposes the task into simpler sub-problems through an iterative cycle of recognition, understanding, and analysis. To enable reactive control, we also design a lightweight adapter for distilling the deliberate focus ability into VLAs. Evaluations on standard embodied AI benchmarks confirm that our method substantially reduces visual hallucinations for both VLMs and VLAs, while preserving computational efficiency in tasks requiring fast execution. Our code and data are released at: https://future-item.github.io/SceneDiver.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
