基于强化学习与生成式预训练模型的四足机器人逼真敏捷性与嬉戏行为
Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Models
AI总结：
本文提出一种分层框架，通过预训练生成式模型与强化学习为四足机器人构建可复用的原始、环境和策略知识，并在MAX机器人上实现了逼真的动物模仿、复杂越障及多智能体追逐游戏中的敏捷策略行为。
摘要：
来自动物和人类的知识启发着机器人创新。人们已通过经典控制器或强化学习方法在四足机器人上实现了大量敏捷运动尝试。这些方法通常依赖物理模型或人工设计的奖励来精确描述特定系统，而非像动物那样依赖一种泛化的理解。本文提出一种分层框架，为腿足机器人构建原始级、环境级和策略级知识，这些知识均可预训练、可复用且可扩充。原始模块从动物运动数据中总结知识，受语言和图像理解领域大型预训练模型的启发，我们引入深度生成模型来产生运动控制信号，激励腿足机器人像真实动物一样行动。然后，我们在更高层次上塑造多种越障能力，通过复用原始模块使其与环境对齐。最后，训练一个策略模块，通过复用前几层的知识，专注于复杂的下游任务。我们将训练好的分层控制器应用于内部研发的四足机器人MAX，使其模仿动物、穿越复杂障碍，并在一个设计好的具有挑战性的多智能体追逐标签游戏中嬉戏，机器人由此展现出逼真的敏捷性和策略。
英文摘要：
Knowledge from animals and humans inspires robotic innovations. Numerous efforts have been made to achieve agile locomotion in quadrupedal robots through classical controllers or reinforcement learning approaches. These methods usually rely on physical models or handcrafted rewards to accurately describe the specific system, rather than on a generalized understanding like animals do. Here we propose a hierarchical framework to construct primitive-, environmental- and strategic-level knowledge that are all pre-trainable, reusable and enrichable for legged robots. The primitive module summarizes knowledge from animal motion data, where, inspired by large pre-trained models in language and image understanding, we introduce deep generative models to produce motor control signals stimulating legged robots to act like real animals. Then, we shape various traversing capabilities at a higher level to align with the environment by reusing the primitive module. Finally, a strategic module is trained focusing on complex downstream tasks by reusing the knowledge from previous levels. We apply the trained hierarchical controllers to the MAX robot, a quadrupedal robot developed in-house, to mimic animals, traverse complex obstacles and play in a designed challenging multi-agent chase tag game, where lifelike agility and strategy emerge in the robots.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
