注意力损失调整的优先经验回放
Attention Loss Adjusted Prioritized Experience Replay
AI总结：
针对优先经验回放（PER）非均匀采样导致Q值估计误差的问题，提出融合改进自注意力网络与双重采样机制的ALAP经验回放算法，在OPENAI gym多类强化学习场景验证了其效率与优势。
摘要：
优先经验回放（Prioritized Experience Replay，PER）是深度强化学习中通过筛选知识量更高的经验样本以提升神经网络训练速率的技术手段。但PER采用的非均匀采样不可避免地会改变状态-动作空间的分布，带来Q值函数的估计误差。本文提出了注意力损失调整优先（Attention Loss Adjusted Prioritized，ALAP）经验回放算法，该算法将改进的自注意力网络与双重采样机制相结合，拟合可调节重要性采样权重的超参数，以消除PER带来的估计误差。为验证算法的有效性与通用性，研究在OPENAI gym环境中，将ALAP与基于值函数、基于策略梯度以及多智能体的强化学习算法分别搭配测试，对比实验验证了所提训练框架的优势与效率。
英文摘要：
Prioritized Experience Replay (PER) is a technical means of deep reinforcement learning by selecting experience samples with more knowledge quantity to improve the training rate of neural network. However, the non-uniform sampling used in PER inevitably shifts the state-action space distribution and brings the estimation error of Q-value function. In this paper, an Attention Loss Adjusted Prioritized (ALAP) Experience Replay algorithm is proposed, which integrates the improved Self-Attention network with Double-Sampling mechanism to fit the hyperparameter that can regulate the importance sampling weights to eliminate the estimation error caused by PER. In order to verify the effectiveness and generality of the algorithm, the ALAP is tested with value-function based, policy-gradient based and multi-agent reinforcement learning algorithms in OPENAI gym, and comparison studies verify the advantage and efficiency of the proposed training framework.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
