arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.09986cs.ROcs.AIcs.LG

在对抗重力的物体操作强化学习中，课程学习比触觉信息的影响更大

Curriculum Is More Influential Than Haptic Information During Reinforcement Learning of Object Manipulation Against Gravity

University of Southern California（南加州大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of California Santa Cruz（加州大学圣克鲁兹分校）；University of Cincinnati（辛辛那提大学）

Pegah Ojaghi, Romina Mir, Ali Marjaninejad, Andrew Erwin, Michael Wehner, Francisco J Valero-Cueva

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究针对无视觉输入的三指仿真手抬球旋球任务，对比课程学习与触觉反馈对强化学习效果的影响，提出新型课程学习率调度器，发现课程选择的作用远大于触觉信息，且无触觉也可完成学习，结果可推广至不同球体。

摘要：

学习用指尖抬起并旋转物体，是实现自主手部内灵巧操作的必要条件。在本研究中，我们探究了多种因素对该任务成功学习策略的影响。具体而言，我们研究了课程学习与触觉反馈在实现灵巧操作学习过程中的作用。我们采用无模型强化学习（Reinforcement Learning, RL），针对无视觉输入的三指仿真机器人手完成对抗重力抬起并旋转球体的任务，对比了不同课程方案与两种触觉信息模式（无触觉感知 vs 3D力传感）的表现。值得注意的是，我们的最优结果通过一种新型的基于课程的学习率调度器获得：该调度器会在奖励函数变更时调整线性衰减的学习率，从而加快向更高奖励水平的收敛速度。我们的研究结果表明，课程的选择会极大地影响灵巧操作不同特征的习得过程。出人意料的是，即使没有触觉反馈也能实现成功学习，这对“触觉信息是灵巧操作任务的必要条件”这一传统假设提出了挑战。我们验证了研究结果可推广至不同重量和尺寸的球体，凸显了我们的学习方法的鲁棒性。因此，本工作强调了课程选择的重要性，并对“自主学习手部内灵巧操作必须依赖触觉信息”这一长期存在的观点提出了质疑。

英文摘要：

Learning to lift and rotate objects with the fingertips is necessary for autonomous in-hand dexterous manipulation. In our study, we explore the impact of various factors on successful learning strategies for this task. Specifically, we investigate the role of curriculum learning and haptic feedback in enabling the learning of dexterous manipulation. Using model-free Reinforcement Learning, we compare different curricula and two haptic information modalities (No-tactile vs. 3D-force sensing) for lifting and rotating a ball against gravity with a three-fingered simulated robotic hand with no visual input. Note that our best results were obtained when we used a novel curriculum-based learning rate scheduler, which adjusts the linearly-decaying learning rate when the reward is changed as it accelerates convergence to higher rewards. Our findings demonstrate that the choice of curriculum greatly biases the acquisition of different features of dexterous manipulation. Surprisingly, successful learning can be achieved even in the absence of tactile feedback, challenging conventional assumptions about the necessity of haptic information for dexterous manipulation tasks. We demonstrate the generalizability of our results to balls of different weights and sizes, underscoring the robustness of our learning approach. This work, therefore, emphasizes the importance of the choice curriculum and challenges long-held notions about the need for tactile information to autonomously learn in-hand dexterous manipulation.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑