MaskVLA：针对视觉-语言-动作模型轨迹过拟合的视觉掩蔽方法
MaskVLA: Visual Masking Against Trajectory Overfitting of Vision-Language-Action Model
另 1 家 · 查看机构详情
- The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
- Ising AI
- University of Glasgow（格拉斯哥大学）
- Southeast University（东南大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
针对VLA模型在有限数据下微调的轨迹过拟合问题，提出基于视觉掩蔽的微调策略MaskVLA，引导模型学习细粒度视觉特征，在RoboTwin 2.0上成功率提升23.2%和16.8%，并在真实机器人上验证有效性。
中文摘要 AI 辅助
视觉-语言-动作（VLA）模型将视觉-语言理解与可执行的机器人动作相结合，实现了机器人控制的端到端学习。然而，我们的实证分析表明，现有模型在有限数据集上进行微调时表现出严重的轨迹过拟合。为了引导模型有效利用腕部相机信息，我们提出了MaskVLA，一种基于掩蔽的微调策略。通过随机掩蔽主相机视觉信息的一小部分，模型被引导自主地学习更细粒度、任务相关且有效的视觉特征。这一过程促使鲁棒策略的出现，从而增强了模型处理复杂操作任务的能力，并提高了其泛化性能。我们的方法已在RoboTwin 2.0上进行了全面评估，与π0和OpenVLA-OFT相比，平均成功率分别提高了23.2%和16.8%。此外，在真实ALOHA机器人上的实验也证明了我们方法的有效性。
英文摘要
Vision-Language-Action (VLA) models integrate vision-language understanding with executable robot actions, enabling end-to-end learning for robot control. However, our empirical analysis reveals that existing models exhibit severe trajectory overfitting when finetuned on limited datasets. To guide the model in effectively utilizing wrist camera information, we propose MaskVLA, a masking-based fine-tuning strategy. By randomly masking a small portion of the main camera's visual information, the model is guided to autonomously learn more fine-grained, task-relevant, and effective visual features. This process leads to the emergence of robust policies, thereby enhancing the model's capability to tackle complex manipulation tasks and improving its generalization performance. Our method has been comprehensively evaluated on RoboTwin 2.0, achieving an average success rate improvement of 23.2% and 16.8% compared to $π_0$ and OpenVLA-OFT, respectively. Furthermore, experiments on real-world ALOHA robots also demonstrate the effectiveness of our approach.
