基于熵控制的内在动机强化学习用于复杂地形四足机器人运动
Entropy-Controlled Intrinsic Motivation Reinforcement Learning for Quadruped Robot Locomotion in Complex Terrains
AI总结：
本文提出基于熵控制的内在动机强化学习算法ECIM，用于提升四足机器人在复杂地形中的运动稳定性与能效。
摘要：
学习是生物和人工系统模仿智能行为的基础。从经典的PPO（近端策略优化）开始，有一系列深度强化学习算法被广泛用于训练四足机器人的运动策略，因为它们的稳定性和样本效率。然而，在所有这些变体中，实验和模拟常常过早收敛，导致运动不优和任务性能下降。因此，在本文中，我们引入了基于熵的强化学习算法，即熵控制内在动机（ECIM），与PPO系列形成对比，通过将内在动机与自适应探索相结合，以减少过早收敛。对于实验，为了与其它基线平行，我们选择在Isaac Gym上应用它，六个地形类别：向上坡、向下坡、不规则粗糙地形、上楼梯、下楼梯和平坦地面，这些是广泛使用的。对于比较，我们的实验始终获得更好的性能：任务奖励增加4-12%，峰值身体pitch振荡减少23-29%，关节加速度减少20-32%，关节扭矩消耗下降11-20%。总体而言，我们的模型ECIM通过结合熵控制和内在动机控制，在不同地形下的四足运动稳定性方面取得了更好的结果，同时减少了能量消耗，使其成为复杂机器人控制任务的实用选择。
英文摘要：
Learning is the basis of both biological and artificial systems when it comes to mimicking intelligent behaviors. From the classical PPO (Proximal Policy Optimization), there is a series of deep reinforcement learning algorithms which are widely used in training locomotion policies for quadrupedal robots because of their stability and sample efficiency. However, among all these variants, experiments and simulations often converge prematurely, leading to suboptimal locomotion and reduced task performance. Therefore, in this paper, we introduce Entropy-Controlled Intrinsic Motivation (ECIM), an entropy-based reinforcement learning algorithm in contrast with the PPO series, that can reduce premature convergence by combining intrinsic motivation with adaptive exploration. For experiments, in order to parallel with other baselines, we chose to apply it in Isaac Gym across six terrain categories: upward slopes, downward slopes, uneven rough terrain, ascending stairs, descending stairs, and flat ground as widely used. For comparison, our experiments consistently achieve better performance: task rewards increase by 4--12%, peak body pitch oscillation is reduced by 23--29%, joint acceleration decreases by 20--32%, and joint torque consumption declines by 11--20%. Overall, our model ECIM, by combining entropy control and intrinsic motivation control, achieves better results in stability across different terrains for quadrupedal locomotion, and at the same time reduces energetic cost and makes it a practical choice for complex robotic control tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
