打破锁定:在低数据VLA后训练中保持可引导性
Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Training
AI总结:
本文提出DeLock方法,通过保留视觉 grounding 和测试时对比提示指导,解决VLA策略在低数据后训练中因过度专业化导致的概念和空间锁定问题,实验证明其优于现有基线方法。
摘要:
您是否曾对一般主义视觉-语言-动作(VLA)策略进行小数据后训练,却发现其不再响应新指令,仅限于后训练期间观察到的行为?我们将其现象定义为锁定:在低数据监督微调(SFT)后,策略过于专业化于后训练数据,无法泛化到新指令,表现为概念锁定(固定于训练对象/属性)和空间锁定(固定于训练空间目标)。许多现有解决方案引入额外监督信号,如来自基础模型或辅助目标的信号,或依赖增强数据集来恢复泛化能力。在本文中,我们表明策略的内部预训练知识已足够:DeLock通过在后训练中保留视觉 grounding 并应用测试时对比提示指导,以引导策略的去噪动态以适应新指令。在八个模拟和现实世界评估中,DeLock一致优于强基线方法,并与使用大量精心编排演示的最先进一般主义策略性能相当。
英文摘要:
Have you ever post-trained a generalist vision-language-action (VLA) policy on a small demonstration dataset, only to find that it stops responding to new instructions and is limited to behaviors observed during post-training? We identify this phenomenon as lock-in: after low-data, supervised fine-tuning (SFT), the policy becomes overly specialized to the post-training data and fails to generalize to novel instructions, manifesting as concept lock-in (fixation on training objects/attributes) and spatial lock-in (fixation on training spatial targets). Many existing remedies introduce additional supervision signals, such as those derived from foundation models or auxiliary objectives, or rely on augmented datasets to recover generalization. In this paper, we show that the policy's internal pre-trained knowledge is sufficient: DeLock mitigates lock-in by preserving visual grounding during post-training and applying test-time contrastive prompt guidance to steer the policy's denoising dynamics according to novel instructions. Across eight simulation and real-world evaluations, DeLock consistently outperforms strong baselines and matches or exceeds the performance of a state-of-the-art generalist policy post-trained with substantially more curated demonstrations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。