arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.04439cs.RO
收藏

$\mathrm{SO}(2)$ -等变强化学习

$\mathrm{SO}(2)$-Equivariant Reinforcement Learning

Northeastern University(东北大学)

Dian Wang, Robin Walters, Robert Platt

摘要:

等变神经网络在其卷积层的结构内强制对称,从而在学习等变或不变函数时大幅提高了样本效率。这样的模型是适用于机器人操作学习,通常可以制定为一个旋转对称的问题。本文在$Q$学习和行动者-评论者强化学习的背景下研究了等变模型结构。我们确定等变和不变的最佳$Q$ -功能和最佳策略的特点,并提出等变DQN和SAC算法,利用这种结构。我们目前的实验表明,我们的等变版本的DQN和SAC可以显着更有效的样本比竞争算法的一类重要的机器人操作问题。

英文摘要:

Equivariant neural networks enforce symmetry within the structure of their convolutional layers, resulting in a substantial improvement in sample efficiency when learning an equivariant or invariant function. Such models are applicable to robotic manipulation learning which can often be formulated as a rotationally symmetric problem. This paper studies equivariant model architectures in the context of $Q$-learning and actor-critic reinforcement learning. We identify equivariant and invariant characteristics of the optimal $Q$-function and the optimal policy and propose equivariant DQN and SAC algorithms that leverage this structure. We present experiments that demonstrate that our equivariant versions of DQN and SAC can be significantly more sample efficient than competing algorithms on an important class of robotic manipulation problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑