arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.09337cs.RO

TORL-VLA:触觉引导的在线强化学习用于接触丰富操作

TORL-VLA: Tactile Guided Online Reinforcement Learning for Contact-Rich Manipulation

Meituan(美团);Beijing Institute of Technology(北京理工大学);Beihang University(北京航空航天大学);State Key Lab of Multimodal Artificial Intelligence Systems, Institute of Automation, CAS(中国科学院自动化研究所多模态人工智能系统国家重点实验室);China University of Mining and Technology (Beijing)(中国矿业大学(北京))

Huaihang Zheng, Yi Yang, Kai Ma, Shenglin Xu, Tian Xie, Guozheng Li, Xiangyu Wang, Yiren Ma, Si Liu, Yinian Mao, Baoxu Liu

AI总结:

提出TORL-VLA框架,结合触觉反馈与在线强化学习,通过触觉导出的力矩感知VLA预测参考动作,并利用轻量在线RL模块优化动作,解决接触条件变化时的策略适应问题,在长时接触任务中提升成功率和执行效率。

摘要:

视觉-语言-动作(VLA)模型已成为机器人操作的有力框架,最近的研究将触觉或力反馈引入VLA以处理接触丰富的任务。然而,这些模型通常作为离线策略部署。当接触条件偏离训练分布时,策略无法进行在线适应,导致接触力不当和重试效率低下等问题。因此,我们提出TORL-VLA,一种触觉引导的在线强化学习框架,将触觉反馈与策略优化相结合用于接触丰富操作。我们的方法引入了一个触觉导出的力矩感知VLA来预测参考动作和未来的力矩序列,同时使用轻量级在线RL模块来优化参考动作。为了稳定地从混合的探索性策略生成和人工干预数据中学习,我们引入了一个干预审查评论家,防止干预后的成功被错误地归因于干预前的策略生成动作。在包括门闩操作、咖啡杯放置和鸡蛋处理等长时接触丰富任务上的真实机器人实验表明,TORL-VLA在子任务和完整任务级别上提高了成功率,并在时间约束的执行效率上优于强基线。

英文摘要:

Vision-Language-Action (VLA) models have become a powerful framework for robotic manipulation, and recent studies have introduced tactile or force feedback into VLAs to address contact-rich tasks. However, these models are typically deployed as offline policies. When contact conditions shift from the training distribution, the policy cannot perform online adaptation, leading to problems such as inappropriate contact forces and inefficient retries. Therefore, we propose TORL-VLA, a tactile-guided online reinforcement learning framework that couples tactile feedback with policy refinement for contact-rich manipulation. Our method introduces a tactile-derived wrench-aware VLA to predict reference actions and future wrench sequences, while a lightweight online RL module is used to refine the reference actions. To stabilize learning from mixed exploratory policy-generated and human-intervention data, we introduce an intervention-censored critic that prevents post-intervention success from being wrongly credited to policy-generated actions preceding intervention. Real-robot experiments on long-horizon contact-rich tasks, including latch manipulation, coffee-cup placement, and egg handling, show that TORL-VLA improves success rates at both subtask and full-task levels, as well as time-bounded execution efficiency over strong baselines. Project page: https://torl-vla.github.io/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑