CAST: 反事实标签提升视觉-语言-动作模型中的指令跟随能力
CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Models
AI总结：
针对VLA模型难以遵循细粒度指令的问题，提出利用视觉语言模型生成反事实标签增强数据集，提升语言基础多样性，实验表明该方法在导航和操作任务中显著提升指令跟随成功率。
摘要：
通用机器人应能理解并遵循用户指令。尽管当前视觉-语言-动作（VLA）模型为将开放词汇语言指令映射到机器人动作提供了强大架构，但它们难以遵循细粒度命令。原因之一是现有机器人数据集缺乏语义多样性和语言基础，特别是对于相似观测缺乏细粒度任务多样性。为解决此问题，我们提出一种新方法，利用视觉语言模型创建反事实标签来增强现有机器人数据集。通过用这些标签增强现有数据集，我们增加了机器人数据集语言基础的多样性和粒度，最终提升了VLA的语言跟随能力。我们通过在3个不同室内外环境中进行视觉语言导航实验，评估了所得模型遵循语言指令的能力，范围从简单的以物体为中心的指令到复杂的指代任务。实验表明，反事实重标记（无需额外数据收集）显著提升了VLA策略的指令跟随能力，超越了最先进方法，并且与在未增强数据上训练的VLA相比，成功率翻倍。我们还评估了该方法在操作VLA上的表现，发现在有干扰物的任务中性能有类似提升。
英文摘要：
Generalist robots should be able to understand and follow user instructions. Despite providing a powerful architecture for mapping open-vocabulary language instructions to robot actions, current vision-language-action (VLA) models struggle to follow fine-grained commands. One cause for this is a lack of semantic diversity and language grounding in existing robot datasets and, specifically, a lack of fine-grained task diversity for similar observations. To address this, we present a novel method to augment existing robot datasets by leveraging vision-language models to create counterfactual labels. By augmenting existing datasets with these labels, we increase the diversity and granularity of language grounding for robot datasets, ultimately improving the language-following capabilities of VLAs. We evaluate the resulting model's ability to follow language instructions, ranging from simple object-centric commands to complex referential tasks, by conducting vision-language navigation experiments in 3 different indoor and outdoor environments. Our experiments show that counterfactual relabeling (without additional data collection) significantly improves instruction-following in VLA policies, outperforming state-of-the-art methods and doubling the success rate compared to VLAs trained on unaugmented data. We also evaluate our method for manipulation VLAs and find a similar gain in performance on tasks with distractors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
