arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.00835cs.ROcs.LG
收藏

CAIMAN:用于样本高效移动-操作的因果动作影响检测

CAIMAN: Causal Action Influence Detection for Sample-efficient Loco-manipulation

ETH Zurich(苏黎世联邦理工学院)

Yuanchen Yuan, Jin Cheng, Núria Armengol Urpí, Stelian Coros

AI总结:

CAIMAN通过因果动作影响作为内在动机目标,提升腿部机器人在稀疏奖励下的样本效率和移动-操作能力。

摘要:

使腿部机器人执行非抓取的移动-操作任务对于提高其通用性至关重要。学习如全身物体推动等行为通常需要复杂的规划策略或大量的任务特定奖励塑造,尤其是在无结构环境中。在本文中,我们提出了CAIMAN,一种实用的强化学习框架,旨在鼓励智能体在环境中获得对其他实体的控制。CAIMAN利用因果动作影响作为内在动机目标,使腿部机器人即使在稀疏任务奖励下也能高效地获得物体推动技能。我们采用分层控制策略,结合低级运动模块和高级策略,该策略生成任务相关的速度命令,并训练以最大化内在奖励。为了估计因果动作影响,我们通过整合运动学先验与训练期间收集的数据来学习环境的动力学。我们实证地展示了CAIMAN在模拟中的优越样本效率和对多样化场景的适应性,以及其成功转移到现实系统而无需进一步微调的能力。一个视频演示可在https://www.youtube.com/watch?v=dNyvT04Cqaw获取。

英文摘要:

Enabling legged robots to perform non-prehensile loco-manipulation is crucial for enhancing their versatility. Learning behaviors such as whole-body object pushing often requires sophisticated planning strategies or extensive task-specific reward shaping, especially in unstructured environments. In this work, we present CAIMAN, a practical reinforcement learning framework that encourages the agent to gain control over other entities in the environment. CAIMAN leverages causal action influence as an intrinsic motivation objective, allowing legged robots to efficiently acquire object pushing skills even under sparse task rewards. We employ a hierarchical control strategy, combining a low-level locomotion module with a high-level policy that generates task-relevant velocity commands and is trained to maximize the intrinsic reward. To estimate causal action influence, we learn the dynamics of the environment by integrating a kinematic prior with data collected during training. We empirically demonstrate CAIMAN's superior sample efficiency and adaptability to diverse scenarios in simulation, as well as its successful transfer to real-world systems without further fine-tuning. A video demo is available at https://www.youtube.com/watch?v=dNyvT04Cqaw.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑