arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.21816cs.RO
收藏

基于自课程模型的可变形线性物体形状控制强化学习

Self-Curriculum Model-based Reinforcement Learning for Shape Control of Deformable Linear Objects

Department of Mechanical Engineering, Tsinghua University(清华大学机械工程系)

Zhaowei Liang, Song Wang, Zhao Jin, Shirui Wu, Dan Wu

AI总结:

提出两阶段框架,结合基于模型强化学习与在线视觉伺服,通过自课程目标生成机制实现可变形线性物体高效精确的形状控制,在仿真和真实任务中优于主流方法。

摘要:

可变形线性物体(DLO)的精确形状控制在工业和医疗等机器人应用中至关重要。然而,现有方法在处理复杂大变形任务(尤其是涉及相反曲率的任务)时面临挑战,且缺乏效率和精度。为此,我们提出一个结合强化学习(RL)和在线视觉伺服的兩阶段框架。在大变形阶段,引入基于模型强化学习方法,使用动力学模型集成显著提高样本效率。此外,我们设计了一种自课程目标生成机制,通过想象评估动态选择具有高多样性的中等难度目标,从而优化策略学习过程。在小变形阶段,部署基于雅可比矩阵的视觉伺服控制器以确保高精度收敛。仿真结果表明,所提方法实现了高效的策略学习,并在形状控制成功率和精度上显著优于主流基线。此外,该框架通过零样本适应将仿真中训练的策略有效迁移到真实任务中。它成功完成了30个案例,涉及不同尺寸和材料的DLO的多样化初始和目标形状。项目网站见:此 https URL

英文摘要:

Precise shape control of Deformable Linear Objects (DLOs) is crucial in robotic applications such as industrial and medical fields. However, existing methods face challenges in handling complex large deformation tasks, especially those involving opposite curvatures, and lack efficiency and precision. To address this, we propose a two-stage framework combining Reinforcement Learning (RL) and online visual servoing. In the large-deformation stage, a model-based reinforcement learning approach using an ensemble of dynamics models is introduced to significantly improve sample efficiency. Additionally, we design a self-curriculum goal generation mechanism that dynamically selects intermediate-difficulty goals with high diversity through imagined evaluations, thereby optimizing the policy learning process. In the small-deformation stage, a Jacobian-based visual servo controller is deployed to ensure high-precision convergence. Simulation results show that the proposed method enables efficient policy learning and significantly outperforms mainstream baselines in shape control success rate and precision. Furthermore, the framework effectively transfers the policy trained in simulation to real-world tasks with zero-shot adaptation. It successfully completes all 30 cases with diverse initial and target shapes across DLOs of different sizes and materials. The project website is available at: https://anonymous.4open.science/w/sc-mbrl-dlo-EB48/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑