面向机器人操作、基于对称性引导示范的样本高效强化学习
Sample-Efficient Reinforcement Learning with Symmetry-Guided Demonstrations for Robotic Manipulation
AI总结：
本文提出Demo-EASE框架，通过通用抽象对称性聚合示范并结合双缓冲区与行为克隆，显著提升机器人操作中强化学习的样本效率和最终性能。
摘要：
强化学习（RL）存在样本效率低的问题，尤其是在复杂机器人操作任务的高维连续状态-动作空间中。即使示范数量有限且来自简化环境，利用先验知识也能提升RL性能。为此，我们定义了通用抽象对称性（General Abstract Symmetry，GAS），用于聚合来自机器人环境对称抽象划分的示范。我们提出Demo-EASE，一种采用双缓冲区架构的新型训练框架，同时存储示范和RL生成的经验。Demo-EASE通过对称性引导的示范和行为克隆提升样本效率，实现强初始化和平衡的探索-利用。Demo-EASE兼容在线策略和离线策略RL算法，支持多种训练模式。我们在PyBullet中使用Kinova Gen3机器人进行关节空间控制的三个仿真实验中评估了该框架。结果表明，与标准RL基线相比，Demo-EASE显著加速收敛并提升最终性能，展示了其在真实世界机器人操作学习中的高效潜力。
英文摘要：
Reinforcement learning (RL) suffers from low sample efficiency, particularly in high-dimensional continuous state-action spaces of complex robotic manipulation tasks. RL performance can improve by leveraging prior knowledge, even when demonstrations are limited and collected from simplified environments. To address this, we define General Abstract Symmetry (GAS) for aggregating demonstrations from symmetrical abstract partitions of the robot environment. We introduce Demo-EASE, a novel training framework using a dual-buffer architecture that stores both demonstrations and RL-generated experiences. Demo-EASE improves sample efficiency through symmetry-guided demonstrations and behavior cloning, enabling strong initialization and balanced exploration-exploitation. Demo-EASE is compatible with both on-policy and off-policy RL algorithms, supporting various training regimes. We evaluate our framework in three simulation experiments using a Kinova Gen3 robot with joint-space control within PyBullet. Our results show that Demo-EASE significantly accelerates convergence and improves final performance compared to standard RL baselines, demonstrating its potential for efficient real-world robotic manipulation learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
