学习多个数量级的价值观
Learning values across many orders of magnitude
摘要：
大多数学习算法对于被近似的函数的尺度不是不变的。我们建议自适应地规范化学习中使用的目标。这在基于值的强化学习中很有用，当我们更新行为策略时，适当的值近似值的大小会随着时间的推移而变化。我们的主要动机是之前学习玩Atari游戏的工作，其中奖励都被限制在预定的范围内。这种裁剪有助于使用单个学习算法在许多不同的游戏中学习，但是裁剪的奖励函数可能导致性质上不同的行为。使用自适应归一化，我们可以在不降低整体性能的情况下删除这种特定于域的启发式算法。
英文摘要：
Most learning algorithms are not invariant to the scale of the function that is being approximated. We propose to adaptively normalize the targets used in learning. This is useful in value-based reinforcement learning, where the magnitude of appropriate value approximations can change over time when we update the policy of behavior. Our main motivation is prior work on learning to play Atari games, where the rewards were all clipped to a predetermined range. This clipping facilitates learning across many different games with a single learning algorithm, but a clipped reward function can result in qualitatively different behavior. Using the adaptive normalization we can remove this domain-specific heuristic without diminishing overall performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
