基于多样性的高效经验回放用于深度强化学习
Efficient Diversity-based Experience Replay for Deep Reinforcement Learning
AI总结：
针对现有经验回放方法在现实高维场景中效率低的问题，提出基于多样性的高效经验回放（EDER），利用行列式点过程、Cholesky分解和拒绝采样，在多个基准上显著提升学习效率与性能。
摘要：
经验回放通过利用过去的经验被广泛用于提高强化学习中的学习效率。然而，现有的经验回放方法，无论是基于均匀采样还是优先采样，通常都存在效率低下的问题，尤其是在具有高维状态空间的现实场景中。为了解决这一局限性，我们提出了一种新颖的方法，即基于多样性的高效经验回放（EDER）。EDER采用行列式点过程来建模样本之间的多样性，并根据样本之间的多样性进行优先回放。为了进一步提高学习效率，我们结合了Cholesky分解来处理现实环境中的大状态空间。此外，应用拒绝采样来选择具有更高多样性的样本，从而提升整体学习效果。我们在MuJoCo中的机器人操作任务、Atari游戏以及Habitat中的真实室内环境上进行了大量实验。结果表明，我们的方法不仅显著提高了学习效率，而且在高维、现实环境中实现了优越的性能。
英文摘要：
Experience replay is widely used to improve learning efficiency in reinforcement learning by leveraging past experiences. However, existing experience replay methods, whether based on uniform or prioritized sampling, often suffer from low efficiency, particularly in real-world scenarios with high-dimensional state spaces. To address this limitation, we propose a novel approach, Efficient Diversity-based Experience Replay (EDER). EDER employs a determinantal point process to model the diversity between samples and prioritizes replay based on the diversity between samples. To further enhance learning efficiency, we incorporate Cholesky decomposition for handling large state spaces in realistic environments. Additionally, rejection sampling is applied to select samples with higher diversity, thereby improving overall learning efficacy. Extensive experiments are conducted on robotic manipulation tasks in MuJoCo, Atari games, and realistic indoor environments in Habitat. The results demonstrate that our approach not only significantly improves learning efficiency but also achieves superior performance in high-dimensional, realistic environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
