深度学习正则化对离线强化学习中Actor的作用
The Role of Deep Learning Regularizations on Actors in Offline RL
AI总结：
本文研究离线强化学习中actor网络泛化瓶颈问题，实验证明对actor网络应用dropout、层归一化等标准正则化技术，可在两种算法和三个D4RL领域上平均提升6%性能。
摘要：
深度学习正则化技术，如dropout、层归一化或权重衰减，在现代人工神经网络的构建中被广泛采用，通常能带来更稳健的训练过程和更好的泛化能力。然而，在强化学习（RL）领域，这些技术的应用较为有限，通常只用于价值函数估计器（Hiraoka et al., 2021; Smith et al., 2022），并且可能产生不利影响。这一问题在离线RL环境中更为突出，离线RL与监督学习更相似，但受到的关注较少。近期连续离线RL方面的工作（Park et al., 2024）表明，尽管我们可以构建足够强大的critic网络，actor网络的泛化仍然是瓶颈。在本研究中，我们通过实验证明，在离线RL actor-critic算法中对actor网络应用标准正则化技术，在两种算法和三个不同的连续D4RL领域上平均带来6%的提升。
英文摘要：
Deep learning regularization techniques, such as dropout, layer normalization, or weight decay, are widely adopted in the construction of modern artificial neural networks, often resulting in more robust training processes and improved generalization capabilities. However, in the domain of Reinforcement Learning (RL), the application of these techniques has been limited, usually applied to value function estimators (Hiraoka et al., 2021; Smith et al., 2022), and may result in detrimental effects. This issue is even more pronounced in offline RL settings, which bear greater similarity to supervised learning but have received less attention. Recent work in continuous offline RL (Park et al., 2024) has demonstrated that while we can build sufficiently powerful critic networks, the generalization of actor networks remains a bottleneck. In this study, we empirically show that applying standard regularization techniques to actor networks in offline RL actor-critic algorithms yields improvements of 6% on average across two algorithms and three different continuous D4RL domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
