Agents trained with deep reinforcement learning algorithms are capable of performing highly complex tasks including locomotion in continuous environments. We investigate transferring the learning acquired in one task to a set of previously unseen tasks. Generalization and overfitting in deep reinforcement learning are not commonly addressed in current transfer learning research. Conducting a comparative analysis without an intermediate regularization step results in underperforming benchmarks and inaccurate algorithm comparisons due to rudimentary assessments. In this study, we propose regularization techniques in deep reinforcement learning for continuous control through the application of sample elimination, early stopping and maximum entropy regularized adversarial learning. First, the importance of the inclusion of training iteration number to the hyperparameters in deep transfer reinforcement learning will be discussed. Because source task performance is not indicative of the generalization capacity of the algorithm, we start by acknowledging the training iteration number as a hyperparameter. In line with this, we introduce an additional step of resorting to earlier snapshots of policy parameters to prevent overfitting to the source task. Then, to generate robust policies, we discard the samples that lead to overfitting via a method we call strict clipping. Furthermore, we increase the generalization capacity in widely used transfer learning benchmarks by using maximum entropy regularization, different critic methods, and curriculum learning in an adversarial setup. Subsequently, we propose maximum entropy adversarial reinforcement learning to increase the domain randomization. Finally, we evaluate the robustness of these methods on simulated robots in target environments where the morphology of the robot, gravity, and tangential friction coefficient of the environment are altered.
Understanding Generalization in Transformers: Error Bounds and Training Dynamics Under Benign and Harmful Overfitting
Zhongtai Securities Institute for Financial Studies, Shandong University（山东大学中泰证券金融研究院）；School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）
Dynamical Decoupling of Generalization and Overfitting in Large Two-Layer Networks
Department of Statistics and Department of Mathematics, Stanford University（统计系和数学系，斯坦福大学）；Université Paris-Saclay, CNRS, CEA, Institut de Physique Théorique, Gif-Sur-Yvette, France（巴黎-萨克雷大学，法国国家科学研究中心，法国原子能机构，理论物理研究所，法国吉夫-索-耶vette）
Friend or Foe? Harnessing Controllable Overfitting for Anomaly Detection
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所基础模型研究中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Objecteye Inc.（Objecteye公司）
MEAT: Median-Ensemble Adversarial Training for Improving Robustness and Generalization
Fujian Province Key Laboratory of Information Security（福建省信息安全重点实验室）；College of Computer Science and Big Data, Fuzhou University（福州大学计算机与大数据学院）；Yuan Ze University（元智大学）
Overfitting has a limitation: a model-independent generalization gap bound based on Rényi entropy
Department of Mathematics Faculty of Science The University of Hong Kong Hong Kong SAR（香港大学数学系）；School of Computing and Mathematical Sciences Faculty of Engineering and Science. University of Greenwich London, United Kingdom（格林威治大学工程与科学学院）
On the Generalization and Causal Explanation in Self-Supervised Learning
Institute of Software Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）；Hong Kong University of Science and Technology（香港科技大学）
Broken Ergodicity and the Violation of the Fluctuation-Dissipation Theorem Lead to Generalization Beyond Overfitting in Machine Learning
Department of Physics, University of California San Diego（加州大学圣地亚哥分校物理系）；Halıcıoğlu Data Science Institute, University of California San Diego（加州大学圣地亚哥分校Halıcıoğlu数据科学研究所）
On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training
Hong Kong University of Science and Technology（香港科技大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；City University of Hong Kong（香港城市大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Quantifying Overfitting: Evaluating Neural Network Performance through Analysis of Null Space
Iran University of Science and Technology（伊朗科技大学）；Okinawa Institute of Science and Technology（冲绳科学技术研究所）；Institute For Research In Fundamental Sciences（基础科学研究所）
