视觉与语言动作政策的贝叶斯分解：看见以行动，提示以指定
Seeing to Act, Prompting to Specify: A Bayesian Factorization of Vision Language Action Policy
AI总结：
本文提出BayesVLA，通过贝叶斯分解策略，解决VLA模型中因模态不平衡导致的灾难性遗忘问题，提升泛化能力和指令遵循性能。
摘要：
在视觉-语言-动作（VLA）模型中，追求分布外泛化通常受到在微调过程中视觉-语言模型（VLM）主干的灾难性遗忘的阻碍。虽然与外部推理数据的协同训练有所帮助，但需要经验性的调优和数据相关开销。除了这些外部依赖，我们发现VLA数据集中存在内在原因：模态不平衡，其中语言多样性远低于视觉和动作多样性。这种不平衡使模型偏向于视觉捷径和语言遗忘。为了解决这个问题，我们引入了BayesVLA，一种贝叶斯分解，将策略分解为视觉-动作先验，支持“看见以行动”，以及语言条件的似然，实现“提示以指定”。这种内在方法保留了泛化能力并促进了指令遵循。我们进一步结合预接触和后接触阶段，以更好地利用预训练的基础模型。信息论分析正式验证了我们的方法在缓解捷径学习方面的有效性。广泛的实验表明，与现有方法相比，其在未见指令、对象和环境上的泛化能力更优。项目页面可在：https://xukechun.github.io/papers/BayesVLA 上找到。
英文摘要：
The pursuit of out-of-distribution generalization in Vision-Language-Action (VLA) models is often hindered by catastrophic forgetting of the Vision-Language Model (VLM) backbone during fine-tuning. While co-training with external reasoning data helps, it requires experienced tuning and data-related overhead. Beyond such external dependencies, we identify an intrinsic cause within VLA datasets: modality imbalance, where language diversity is much lower than visual and action diversity. This imbalance biases the model toward visual shortcuts and language forgetting. To address this, we introduce BayesVLA, a Bayesian factorization that decomposes the policy into a visual-action prior, supporting seeing-to-act, and a language-conditioned likelihood, enabling prompt-to-specify. This inherently preserves generalization and promotes instruction following. We further incorporate pre- and post-contact phases to better leverage pre-trained foundation models. Information-theoretic analysis formally validates our effectiveness in mitigating shortcut learning. Extensive experiments show superior generalization to unseen instructions, objects, and environments compared to existing methods. Project page is available at: https://xukechun.github.io/papers/BayesVLA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
