一种受中枢运动系统启发的预训练强化学习方法用于机器人控制
A Central Motor System Inspired Pre-training Reinforcement Learning for Robotic Control
AI总结：
本文提出受中枢运动系统启发的预训练强化学习方法CMS-PRL，通过融合奖励机制、基底神经节式技能编码和技能活动函数生成多样可复用技能，在稀疏奖励任务中优于基线方法，尤其适用于高自由度机器人。
摘要：
智能机器人的发展需要能够应对动态环境和不断演变任务的控制策略。预训练强化学习已成为满足这些需求的有效方法，它使机器人能够获得可复用的运动技能。然而，这些方法通常依赖大型数据集或专家设计的目标空间，限制了适应性。此外，这些方法难以在高维状态空间中生成动态且多样的技能，从而降低了对下游任务的有效性。本文提出CMS-PRL，一种受中枢运动系统（Central Motor System, CMS）启发的预训练强化学习方法。首先，我们引入一种融合奖励机制，将基础运动奖励与互信息奖励相结合，在预训练过程中促进动态技能的发现，且不依赖外部数据。其次，我们设计了一种受基底神经节运动程序启发的技能编码方法，在预训练期间提供丰富且连续的技能指令。最后，我们提出一种技能活动函数来调节运动技能活动，使机器人能够生成具有不同活动水平的技能，从而增强机器人在下游任务中的灵活性。我们在四类机器人上的一组具有挑战性的稀疏奖励任务中评估了该模型。实验结果表明，CMS-PRL生成了多样化、可复用的运动技能，以解决各种下游任务，并优于基线方法，尤其是在高自由度机器人和复杂任务中。
英文摘要：
The development of intelligent robots requires control policies that can handle dynamic environments and evolving tasks. Pre-training reinforcement learning has emerged as an effective approach to address these demands by enabling robots to acquire reusable motor skills. However, they often rely on large datasets or expert-designed goal spaces, limiting adaptability. Additionally, these methods need help to generate dynamic and diverse skills in high-dimensional state spaces, reducing their effectiveness for downstream tasks. In this paper, we propose CMS-PRL, a pre-training reinforcement learning method inspired by the Central Motor System (CMS). First, we introduce a fusion reward mechanism that combines the basic motor reward with mutual information reward, promoting the discovery of dynamic skills during pre-training without reliance on external data. Second, we design a skill encoding method inspired by the motor program of the basal ganglia, providing rich and continuous skill instructions during pre-training. Finally, we propose a skill activity function to regulate motor skill activity, enabling the generation of skills with different activity levels, thereby enhancing the robot's flexibility in downstream tasks. We evaluate the model on four types of robots in a challenging set of sparse-reward tasks. Experimental results demonstrate that CMS-PRL generates diverse, reusable motor skills to solve various downstream tasks and outperforms baseline methods, particularly in high-degree-of-freedom robots and complex tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
