arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.06355cs.AI
收藏

强化学习中用于泛化的局部特征交换

Local Feature Swapping for Generalization in Reinforcement Learning

IRT Saint-Exupéry(圣埃克苏佩里跨行业技术研究中心);ISAE-SUPAERO(法国高等航空航天学院);ANITI(图卢兹人工智能与自然智能研究所);Université de Toulouse(图卢兹大学)

David Bertoin, Emmanuel Rachelson

摘要:

在过去的几年里,计算资源和深度学习研究的加速发展已经在一系列任务中取得了重大的实际成功,特别是在计算机视觉方面。在这些进步的基础上,强化学习也出现了飞跃,出现了能够直接从视觉观察中做出决策的智能体。尽管取得了这些成功,但神经架构的过度参数化导致对训练期间使用的数据的记忆,从而缺乏泛化能力。基于视觉输入的强化学习代理也会因错误地将奖励与背景元素等不相关的视觉特征相关联而遭受这种现象。为了缓解这个问题,我们引入了一种新的正则化技术,包括通道一致的局部置换(CLOP)的特征图。所提出的排列诱导空间相关性的鲁棒性,并有助于防止RL中的过拟合行为。我们在OpenAI Procgen Benchmark上证明,使用CLOP方法训练的RL代理对视觉变化表现出鲁棒性,并且比使用其他最先进的正则化技术训练的代理具有更好的泛化特性。我们还证明了CLOP作为监督学习中的一般正则化技术的有效性。

英文摘要:

Over the past few years, the acceleration of computing resources and research in deep learning has led to significant practical successes in a range of tasks, including in particular in computer vision. Building on these advances, reinforcement learning has also seen a leap forward with the emergence of agents capable of making decisions directly from visual observations. Despite these successes, the over-parametrization of neural architectures leads to memorization of the data used during training and thus to a lack of generalization. Reinforcement learning agents based on visual inputs also suffer from this phenomenon by erroneously correlating rewards with unrelated visual features such as background elements. To alleviate this problem, we introduce a new regularization technique consisting of channel-consistent local permutations (CLOP) of the feature maps. The proposed permutations induce robustness to spatial correlations and help prevent overfitting behaviors in RL. We demonstrate, on the OpenAI Procgen Benchmark, that RL agents trained with the CLOP method exhibit robustness to visual changes and better generalization properties than agents trained using other state-of-the-art regularization techniques. We also demonstrate the effectiveness of CLOP as a general regularization technique in supervised learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑