看得更远且更聪明：基于价值引导的多路径反射用于VLM策略优化
Seeing Farther and Smarter: Value-Guided Multi-Path Reflection for VLM Policy Optimization
AI总结：
本研究提出了一种基于价值引导的多路径反射方法，用于提升VLM在机器人操作任务中的策略优化性能，通过解耦状态评估与动作生成，提高决策鲁棒性并减少推理时间。
摘要：
解决复杂的、长周期的机器人操作任务需要深入理解物理相互作用，对它们的长期后果进行推理，并进行精确的高层规划。视觉-语言模型（VLMs）为实现这一目标提供了一个通用的感知-推理-行动框架。然而，先前使用反思规划来指导VLMs修正动作的方法存在显著限制。这些方法依赖于从噪声前瞻预测中学习状态值的低效且往往不准确的隐式学习，只评估单一的贪婪未来，并且面临显著的推理延迟。为了解决这些限制，我们提出了一种新的测试时间计算框架，将状态评估与动作生成解耦。这为鲁棒决策提供了更直接和细致的监督信号。我们的方法显式建模动作计划的优势，通过其减少目标距离的量来量化，并使用可扩展的批评者进行估计。为了解决单轨迹评估的随机性，我们采用束搜索来探索多条未来路径，并在解码过程中聚合它们以建模其预期的长期回报，从而产生更鲁棒的动作生成。此外，我们引入了一个轻量级、基于置信度的触发器，允许在直接预测可靠时提前退出，仅在必要时调用反思。在多样、未见过的多阶段机器人操作任务上的广泛实验显示，我们的方法在状态-of-the-art基线上的成功率提高了24.6%，同时显著减少了推理时间，减少了56.5%。
英文摘要：
Solving complex, long-horizon robotic manipulation tasks requires a deep understanding of physical interactions, reasoning about their long-term consequences, and precise high-level planning. Vision-Language Models (VLMs) offer a general perceive-reason-act framework for this goal. However, previous approaches using reflective planning to guide VLMs in correcting actions encounter significant limitations. These methods rely on inefficient and often inaccurate implicit learning of state-values from noisy foresight predictions, evaluate only a single greedy future, and suffer from substantial inference latency. To address these limitations, we propose a novel test-time computation framework that decouples state evaluation from action generation. This provides a more direct and fine-grained supervisory signal for robust decision-making. Our method explicitly models the advantage of an action plan, quantified by its reduction in distance to the goal, and uses a scalable critic to estimate. To address the stochastic nature of single-trajectory evaluation, we employ beam search to explore multiple future paths and aggregate them during decoding to model their expected long-term returns, leading to more robust action generation. Additionally, we introduce a lightweight, confidence-based trigger that allows for early exit when direct predictions are reliable, invoking reflection only when necessary. Extensive experiments on diverse, unseen multi-stage robotic manipulation tasks demonstrate a 24.6% improvement in success rate over state-of-the-art baselines, while significantly reducing inference time by 56.5%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
