等变离线强化学习
Equivariant Offline Reinforcement Learning
AI总结：
针对机器人操作中样本效率低的问题，该研究将$SO(2)$-等变神经网络应用于低数据量的离线RL，验证了等变CQL和IQL算法性能更优，为等变性提升低数据场景下的离线学习效果提供了实证支持。
摘要：
由于收集专家演示的成本高昂，且通过在线强化学习（RL）在机器人上进行策略学习存在诸多挑战，因此将基于学习的方法应用于机器人操作时，样本效率至关重要。离线RL通过支持利用任意行为策略（无论其质量如何）收集的离线数据集进行策略学习，解决了这一问题。然而，近期离线RL的进展主要聚焦于从大型数据集学习。鉴于许多机器人操作任务可被建模为旋转对称问题，我们研究了$SO(2)$-等变神经网络在仅有少量演示的离线RL中的应用。实验结果表明，保守Q学习（Conservative Q-Learning, CQL）和隐式Q学习（Implicit Q-Learning, IQL）的等变版本性能优于其非等变对应版本。我们提供了实证证据，证明等变性如何在低数据 regime 下提升离线学习算法的性能。
英文摘要：
Sample efficiency is critical when applying learning-based methods to robotic manipulation due to the high cost of collecting expert demonstrations and the challenges of on-robot policy learning through online Reinforcement Learning (RL). Offline RL addresses this issue by enabling policy learning from an offline dataset collected using any behavioral policy, regardless of its quality. However, recent advancements in offline RL have predominantly focused on learning from large datasets. Given that many robotic manipulation tasks can be formulated as rotation-symmetric problems, we investigate the use of $SO(2)$-equivariant neural networks for offline RL with a limited number of demonstrations. Our experimental results show that equivariant versions of Conservative Q-Learning (CQL) and Implicit Q-Learning (IQL) outperform their non-equivariant counterparts. We provide empirical evidence demonstrating how equivariance improves offline learning algorithms in the low-data regime.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
