SLAC：通过无监督模拟预训练实现安全高效的真实机器人强化学习
SLAC: Safe and Efficient Real-Robot Reinforcement Learning via Unsupervised Simulation Pre-Training
AI总结：
研究旨在解决高自由度机器人强化学习难题，提出SLAC方法，利用低保真模拟器预训练潜在动作空间，通过定制无监督方法训练并用于新型离策略算法，在双手移动操纵任务中达领先性能，高效学习复杂任务。
摘要：
构建家用和工业机器人需要掌握对移动操纵器等高自由度系统的控制。强化学习虽有望自主获取机器人控制策略，但扩展到高自由度系统仍具挑战。直接在现实世界中进行强化学习需兼顾安全探索和高样本效率，难以实现。而模拟到现实的强化学习因现实差距往往很脆弱。本文介绍了SLAC方法，通过利用低保真模拟器预训练任务无关的潜在动作空间，使复杂实体的现实世界强化学习可行。SLAC通过定制的无监督技能发现方法训练潜在动作空间，促进时间抽象、解缠和安全，便于高效下游学习。学习到潜在动作空间后，SLAC将其用作新型离策略强化学习算法的动作接口，通过现实世界交互自主学习下游任务。在一系列双手移动操纵任务上评估SLAC，其取得了领先性能。特别是，SLAC在不到一小时的现实世界交互中就能学习富含接触的全身任务，且无需依赖任何演示或手工行为先验知识。
英文摘要：
Building capable household and industrial robots requires mastering the control of versatile, high-degree-of-freedom (DoF) systems such as mobile manipulators. While reinforcement learning (RL) holds promise for autonomously acquiring robot control policies, scaling it to high-DoF embodiments remains challenging. Direct RL in the real world demands both safe exploration and high sample efficiency, which are difficult to achieve in practice. Sim-to-real RL, on the other hand, is often brittle due to the reality gap. This paper introduces SLAC, a method that renders real-world RL feasible for complex embodiments by leveraging a low-fidelity simulator to pretrain a task-agnostic latent action space. SLAC trains this latent action space via a customized unsupervised skill discovery method designed to promote temporal abstraction, disentanglement, and safety, thereby facilitating efficient downstream learning. Once a latent action space is learned, SLAC uses it as the action interface for a novel off-policy RL algorithm to autonomously learn downstream tasks through real-world interactions. We evaluate SLAC against existing methods on a suite of bimanual mobile manipulation tasks, where it achieves state-of-the-art performance. Notably, SLAC learns contact-rich whole-body tasks in under an hour of real-world interactions, without relying on any demonstrations or hand-crafted behavior priors. More information and robot videos at robo-rl.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
