拿来苹果,别拿沙发:具身AI指令中无关上下文对VLA模型的影响
Bring the Apple, Not the Sofa: Impact of Irrelevant Context in Embodied AI Commands on VLA Models
AI总结:
本研究系统评估了VLA模型在指令噪声(人类释义和无关上下文)下的鲁棒性,发现语义相似的上下文导致性能大幅下降,并提出LLM过滤框架以恢复至多98.5%的原始性能。
摘要:
视觉语言动作(VLA)模型在具身AI中被广泛使用,使机器人能够解释并执行语言指令。然而,它们在真实场景中对自然语言变异的鲁棒性尚未得到深入研究。在这项工作中,我们对最先进的VLA模型在语言扰动下的鲁棒性进行了一项新颖的系统性研究。具体而言,我们在两种指令噪声下评估模型性能:(1)人类生成的释义和(2)添加无关上下文。我们进一步根据无关上下文的长度及其与机器人命令的语义和词汇接近程度,将其分为两组。在这项研究中,我们观察到随着上下文规模的扩大,性能持续下降。我们还证明,模型对随机上下文可以表现出相对的鲁棒性,性能下降在10%以内,而相同长度的语义和词汇相似的上下文可能引发约50%的质量下降。人类对指令的释义导致性能下降近20%。为了缓解这一问题,我们提出了一种基于LLM的过滤框架,从噪声输入中提取核心命令。在噪声条件下,加入我们的过滤步骤使模型能够恢复其原始性能的至多98.5%。
英文摘要:
Vision Language Action (VLA) models are widely used in Embodied AI, enabling robots to interpret and execute language instructions. However, their robustness to natural language variability in real-world scenarios has not been thoroughly investigated. In this work, we present a novel systematic study of the robustness of state-of-the-art VLA models under linguistic perturbations. Specifically, we evaluate model performance under two types of instruction noise: (1) human-generated paraphrasing and (2) the addition of irrelevant context. We further categorize irrelevant contexts into two groups according to their length and their semantic and lexical proximity to robot commands. In this study, we observe consistent performance degradation as context size expands. We also demonstrate that the model can exhibit relative robustness to random context, with a performance drop within 10%, while semantically and lexically similar context of the same length can trigger a quality decline of around 50%. Human paraphrases of instructions lead to a drop of nearly 20%. To mitigate this, we propose an LLM-based filtering framework that extracts core commands from noisy inputs. Incorporating our filtering step allows models to recover up to 98.5% of their original performance under noisy conditions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。