基于极其简单的模态掩码机制的鲁棒双臂视觉-语言-动作模型
Robust Bimanual Vision-Language-Action Models via Embarrassingly Simple Modality Masking
浏览论文内容
中文总结 AI 辅助
该研究针对双臂操作中查询式VLA模型的动作不连续问题，提出仅训练用的模态掩码机制（M3），在RoboTwin 2.0等任务上使平均成功率提升超11.4%至30%以上，有效增强了模型鲁棒性。
中文摘要 AI 辅助
基于查询的视觉-语言-动作（VLA）模型提供低延迟推理，对双臂机器人操作具有吸引力，但我们观察到它们在复杂双臂任务中仍会出现动作不连续和执行失败的情况。我们假设不稳定的多视图与语言融合是这些失败的促成因素之一，常伴随注意力扩散到干扰区域。为提升鲁棒性，我们引入模态掩码机制（M3），这是一种极其简单的仅训练策略，无需架构改动或大规模机器人预训练。M3在训练期间随机掩码部分模态通道，让策略接触受控的部分观测，鼓励其减少对干扰线索的依赖，更多依赖可靠证据。我们在RoboTwin 2.0的10个双臂任务和3个长 horizon 真实世界任务上评估M3。与Adapter基线相比，M3在Clean设置中平均成功率提升21.7%，在Clean2Rand（策略在干净演示上训练，在随机场景上评估）中提升11.4%，同时真实世界全任务平均成功率提升超30%。这些结果表明，结构化训练时掩码是提升双臂操作查询式VLA策略鲁棒性的实用方法。
英文摘要
Query-based Vision-Language-Action (VLA) models offer low-latency inference that is attractive for bimanual robotic manipulation, but we observe that they can still exhibit discontinuous actions and execution failures in complex dual-arm tasks. We hypothesize that unstable multi-view and language fusion is one contributing factor in these failures, often coinciding with attention spreading to distracting regions. To improve robustness, we introduce the Modality Masking Mechanism (M3), an embarrassingly simple, training-only strategy that requires no architectural changes or large-scale robot pretraining. M3 stochastically masks subsets of modality channels during training, exposing the policy to controlled partial observations and encouraging it to rely less on distracting cues and more on evidence that remains reliable. We evaluate M3 on ten bimanual tasks from RoboTwin 2.0 and on three long-horizon real-world tasks. Compared with the Adapter baseline, M3 improves average success by 21.7% in the Clean setting and 11.4% in Clean2Rand, where policies are trained on clean demonstrations and evaluated on randomized scenes, while also improving averaged real-world full-task success by over 30%. These results suggest that structured training-time masking is a practical way to improve the robustness of query-based VLA policies for bimanual manipulation.
发表机构
- Shanghai Innovation Institute（上海创新研究院）
- Wuhan University（武汉大学）
- Zhejiang University（浙江大学）
- Shanghai Jiao Tong University（上海交通大学）
- University of Science and Technology of China（中国科学技术大学）
- City University of Hong Kong（香港城市大学）
- Southeast University（东南大学）
机构由 AI 辅助整理，请以论文原文为准。
