立场:视觉-语言-动作模型无法被验证执行物理推理
Position: Vision-Language-Action Models Cannot Be Verified to Perform Physical Reasoning
AI总结:
本文指出基于预训练视觉-语言模型的VLA系统在机器人操作基准上的性能提升无法区分语义匹配与物理泛化,现有评估指标不能验证物理推理能力。
摘要:
视觉-语言-动作(VLA)系统建立在预训练的视觉-语言模型(VLM)之上,在机器人操作基准上表现出快速提升的性能。这些提升通常被解释为从互联网规模数据中学习的语义表示能够迁移到物理执行泛化。本文立场认为,这种解释所依据的假设——语义泛化足以支持物理动作决策——尚未得到独立验证,并且无法在当前评估协议下进行测试。我们通过将VLA策略分解为语义映射和物理动作决策,并证明任务成功率(主要评估指标)无法区分这两种能力来源来支持这一观点。因此,基准性能的提升与多种竞争性解释一致,包括语义匹配、分布重叠和真正的物理泛化。我们进一步认为,这种可识别性差距通过叙事漂移被强化,即后续系统继承并强化了先前对性能提升的解释,而没有隔离潜在的因果机制。为解决这一局限性,我们提出一个基于评估设计的研究方向,通过引入受控变化来分别测量语义和物理泛化。这种设计使得无需访问模型内部即可因果归因性能,并能够实证评估VLM骨干作为语义接口而非隐式物理能力来源的作用。我们的目标不是反驳VLM在机器人学中的作用,而是阐明在何种条件下可以有意义地评估物理泛化的主张。
英文摘要:
Vision-Language-Action (VLA) systems, built on pretrained vision-language models (VLMs), have shown rapidly improving performance on robot manipulation benchmarks. These gains are commonly interpreted as evidence that semantic representations learned from internet-scale data transfer to physical execution generalization. This position paper argues that the assumption underlying this interpretation -- that semantic generalization is sufficient to support physical action decisions -- has not been independently verified and cannot be tested under current evaluation protocols. We support this claim by decomposing VLA policies into semantic mapping and physical action decision, and showing that task success rate -- the dominant evaluation metric -- cannot distinguish between these two sources of capability. As a result, improvements in benchmark performance are consistent with multiple competing explanations, including semantic matching, distributional overlap, and genuine physical generalization. We further argue that this identifiability gap has been reinforced through narrative drift, whereby successive systems inherit and strengthen prior interpretations of performance gains without isolating the underlying causal mechanism. To address this limitation, we propose a research direction based on evaluation designs that introduce controlled variation to separately measure semantic and physical generalization. Such designs make it possible to causally attribute performance without requiring access to model internals, and to empirically assess the role of VLM backbones as semantic interfaces rather than implicit sources of physical competence. Our goal is not to refute the role of VLMs in robotics, but to clarify the conditions under which claims of physical generalization can be meaningfully evaluated.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。