arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.01331cs.ROcs.LG
收藏

RobustVLA: 为视觉-语言-动作模型引入鲁棒性感知的强化学习后训练

RobustVLA: Robustness-Aware Reinforcement Post-Training for Vision-Language-Action Models

Westlake University(西湖大学)

Hongyin Zhang, Shuo Zhang, Junxi Jin, Qixin Zeng, Runze Li, Donglin Wang

AI总结:

RobustVLA通过引入鲁棒性感知的强化学习后训练方法,提升视觉-语言-动作模型在环境不确定性下的鲁棒性和可靠性。

摘要:

视觉-语言-动作(VLA)模型最近涌现出作为机器人操作强大通用策略的潜力,受益于大规模多模态预训练。然而,它们在分布外部署中往往无法可靠地泛化,其中不可避免的干扰如观测噪声、传感器误差或执行扰动变得普遍。尽管最近基于强化学习(RL)的后训练提供了一种实用的手段来适应预训练的VLA模型,现有方法主要强调奖励最大化,而忽视了对环境不确定性的鲁棒性。在本工作中,我们引入了RobustVLA,一种轻量级的在线RL后训练方法,旨在显式增强VLA模型的韧性。通过系统的鲁棒性分析,我们识别出两个关键正则化:雅可比正则化,用于缓解对观测噪声的敏感性,以及平滑正则化,用于在动作扰动下稳定策略。在多样化的机器人环境中进行的广泛实验表明,RobustVLA在鲁棒性和可靠性方面显著优于现有最先进的方法。我们的结果强调了原则性鲁棒性感知的强化学习后训练作为提高VLA模型可靠性和鲁棒性关键步骤的重要性。

英文摘要:

Vision-Language-Action (VLA) models have recently emerged as powerful general-purpose policies for robotic manipulation, benefiting from large-scale multi-modal pre-training. However, they often fail to generalize reliably in out-of-distribution deployments, where unavoidable disturbances such as observation noise, sensor errors, or actuation perturbations become prevalent. While recent Reinforcement Learning (RL)-based post-training provides a practical means to adapt pre-trained VLA models, existing methods mainly emphasize reward maximization and overlook robustness to environmental uncertainty. In this work, we introduce RobustVLA, a lightweight online RL post-training method designed to explicitly enhance the resilience of VLA models. Through a systematic robustness analysis, we identify two key regularizations: Jacobian regularization, which mitigates sensitivity to observation noise, and smoothness regularization, which stabilizes policies under action perturbations. Extensive experiments across diverse robotic environments demonstrate that RobustVLA significantly outperforms prior state-of-the-art methods in robustness and reliability. Our results highlight the importance of principled robustness-aware RL post-training as a key step toward improving the reliability and robustness of VLA models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑