ELO评级的序列奖励：推进强化学习模型
ELO-Rated Sequence Rewards: Advancing Reinforcement Learning Models
AI总结：
针对长期强化学习中状态-动作对奖励分配困难的问题，提出基于ELO评级的强化学习算法ERRL，利用专家轨迹偏好计算ELO评级作为奖励，并引入奖励重分配算法缓解训练不稳定性，在5000步长期场景中优于现有基线。
摘要：
强化学习（RL）严重依赖于奖励函数的精心设计。然而，在长期强化学习（LTRL）任务中，为每个状态-动作对准确分配奖励仍然是一个重大挑战。因此，RL智能体通常在专家指导下进行训练。受经济学中序数效用理论的启发，我们提出了一种新颖的奖励估计算法：基于ELO评级的强化学习（ERRL）。该方法有两个关键贡献。首先，它使用轨迹上的专家偏好而非基数奖励（效用）来计算每条轨迹的ELO评级作为其奖励。其次，引入了一种新的奖励重分配算法，以缓解在没有固定锚定奖励情况下的训练不稳定性。在传统RL算法难以应对的长期场景（长达5000步）中，我们的方法优于几个最先进的基线方法。此外，我们还对专家偏好如何影响结果进行了全面分析。
英文摘要：
Reinforcement Learning (RL) heavily relies on the careful design of the reward function. However, accurately assigning rewards to each state-action pair in Long-Term Reinforcement Learning (LTRL) tasks remains a significant challenge. As a result, RL agents are often trained under expert guidance. Inspired by the ordinal utility theory in economics, we propose a novel reward estimation algorithm: ELO-Rating based Reinforcement Learning (ERRL). This approach features two key contributions. First, it uses expert preferences over trajectories rather than cardinal rewards (utilities) to compute the ELO rating of each trajectory as its reward. Second, a new reward redistribution algorithm is introduced to alleviate training instability in the absence of a fixed anchor reward. In long-term scenarios (up to 5000 steps), where traditional RL algorithms struggle, our method outperforms several state-of-the-art baselines. Additionally, we conduct a comprehensive analysis of how expert preferences influence the results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
