发表机构
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对VLA模型在未演示技能组合上泛化失败的问题，提出CRAFT方法，利用可复用技能表示将已演示监督迁移至反事实对，在模拟和真实机器人上提升组合泛化成功率。
AI 中文摘要
视觉-语言-动作（VLA）模型在面对微调演示中未出现的技能组合时，往往难以实现泛化，即使每个组成技能都已被演示过。我们聚焦于一种失败模式——视觉捷径：在微调过程中，视觉观察可以作为指令的代理，因此策略可能执行与相似观察相关联的已演示组合，而非指令指定的组合。这促使我们使用反事实对进行训练，即保持演示观察不变，同时将指令改为指定一个未演示的组合。然而，这些反事实对缺乏对应的已演示动作目标。关键在于，当前所需的技能已被演示过，但这些执行中的动作不能直接作为目标，因为同一技能在不同观察下可能需要不同的动作。我们提出了CRAFT方法，通过可跨同一技能多次执行复用的技能表示，将所需技能的已演示执行监督迁移到反事实对上。在三个VLA模型和两个模拟基准上，CRAFT在未演示组合上提升了成功率，同时保持了已演示组合的高成功率；它还在真实机器人上提升了组合泛化能力。项目网站：此https URL
英文摘要
Vision-language-action (VLA) models often struggle to generalize to skill combinations absent from their fine-tuning demonstrations, even when every constituent skill has been demonstrated. We focus on a vision shortcut as one failure mode: during fine-tuning, visual observations can serve as a proxy for the instruction, so a policy may execute a demonstrated combination associated with similar observations rather than the instructed combination. This motivates training with counterfactual pairs formed by holding a demonstration observation fixed while changing the instruction to specify an undemonstrated combination. These pairs, however, lack corresponding demonstrated action targets. Crucially, the currently required skill has already been demonstrated, but actions from those executions cannot serve as direct targets because the same skill can require different actions across observations. We propose CRAFT, which transfers supervision from demonstrated executions of the required skill to counterfactual pairs using skill representations that can be reused across executions of the same skill. Across three VLA models and two simulation benchmarks, CRAFT improves success on undemonstrated combinations while maintaining high success on demonstrated ones; it also improves compositional generalization on a real robot. Project website: https://taegeunyang.github.io/craft/
Comments26 pages, 5 figures. Project page: https://taegeunyang.github.io/craft/
