发表机构
The University of Tokyo; AI Center, Graduate School of Information Science and Technology, The University of Tokyo（东京大学; 东京大学信息科学与技术研究生院人工智能中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对腿部机器人强化学习中关节空间动作探索能力有限的问题，提出WARL方法，结合扳手引导探索与课程机制。实验表明该方法能让四足机器人在多样地形和任务中稳健学习，验证了切换课程有效性，指出按机器人结构设计扳手探索是未来挑战。
AI 中文摘要
虽然腿部机器人的强化学习已实现高运动性能，但受限于关节空间内动作的有限探索能力。为解决此问题，本研究提出一种新方法——扳手增强强化学习（WARL），将扳手（力和扭矩）引入动作空间。该方法结合扳手引导探索与基于成功率的课程机制，在学习早期扩展探索能力，最终仅通过关节控制获取行为。使用四足机器人的实验表明，WARL能在不同地形和运动任务中稳健学习，无需特定地形奖励调整或复杂课程设计。此外，消融研究验证了逐渐消除扳手的切换课程的有效性。另一方面，研究表明引入扳手会鼓励未充分利用机器人物理实体的行为。这些发现表明，基于扳手的探索增强对提高学习效率有效，但以与机器人物理结构一致的方式设计是未来的关键挑战。
英文摘要
While reinforcement learning for legged robots has achieved high motor performance, it has been constrained by the limited exploration capability of actions confined to the joint space. To address this issue, this study proposes a new method, Wrench-Augmented Reinforcement Learning (WARL), which introduces a wrenche (force and torque) into the action space. The proposed method combines wrench-guided exploration with a success rate-based curriculum mechanism to expand exploration capabilities in the early stages of learning, with the ultimate goal of acquiring behaviors based solely on joint control. Experiments using a quadruped robot demonstrated that WARL can learn robustly across diverse terrains and motor tasks without requiring terrain-specific reward adjustments or complex curriculum designs. Furthermore, an ablation study verified the effectiveness of the Switching Curriculum, which gradually eliminates the wrench. On the other hand, we also show that introducing a wrench can encourage behaviors that do not sufficiently exploit the robot's physical embodiment. These findings suggest that while wrench-based exploration enhancement is effective for improving learning efficiency, designing it in a way that is consistent with the robot's physical structure is a critical future challenge.
CommentsAccepted at IROS 2026, website-https://keitayoneda.github.io/kleiyn-warl/, youtube-https://youtu.be/l-drMNncQp0
