SLIM：基于多评论家的技能学习
SLIM: Skill Learning with Multiple Critics
AI总结：
针对机器人操作中互信息最大化难以产生有用安全行为的问题，提出多评论家技能学习方法SLIM，在actor-critic框架中优雅组合多奖励函数以克服干扰，在桌面操作中显著超越基线方法。
摘要：
自监督技能学习旨在获取利用环境潜在动态的有用行为。基于互信息最大化的潜变量模型在此任务中取得了成功，但在机器人操作场景中仍面临困难。由于这要求影响构成环境的可能庞大的自由度集合，仅靠互信息最大化无法产生有用且安全的操作行为。此外，通过简单组合将额外奖励增补到技能发现奖励中来解决此问题，可能无法产生预期行为。为解决该局限，我们提出了SLIM，一种用于技能发现的多评论家学习方法，特别聚焦于机器人操作。我们的核心见解是，在actor-critic框架中利用多个评论家来优雅地组合多个奖励函数，能显著改进机器人操作的潜变量技能发现，同时克服奖励间可能出现的阻碍收敛至有用技能的干扰。此外，在桌面操作场景中，我们展示了新型技能发现方法的适用性，以分层强化学习方式获取安全高效的 motor primitives，并通过规划利用它们，显著超越了技能发现的基线方法。
英文摘要：
Self-supervised skill learning aims to acquire useful behaviors that leverage the underlying dynamics of the environment. Latent variable models, based on mutual information maximization, have been successful in this task but still struggle in the context of robotic manipulation. As it requires impacting a possibly large set of degrees of freedom composing the environment, mutual information maximization fails alone in producing useful and safe manipulation behaviors. Furthermore, tackling this by augmenting skill discovery rewards with additional rewards through a naive combination might fail to produce desired behaviors. To address this limitation, we introduce SLIM, a multi-critic learning approach for skill discovery with a particular focus on robotic manipulation. Our main insight is that utilizing multiple critics in an actor-critic framework to gracefully combine multiple reward functions leads to a significant improvement in latent-variable skill discovery for robotic manipulation while overcoming possible interference occurring among rewards which hinders convergence to useful skills. Furthermore, in the context of tabletop manipulation, we demonstrate the applicability of our novel skill discovery approach to acquire safe and efficient motor primitives in a hierarchical reinforcement learning fashion and leverage them through planning, significantly surpassing baseline approaches for skill discovery.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
