An agent with an inaccurate model of its environment faces a difficult choice: it can ignore the errors in its model and act in the real world in whatever way it determines is optimal with respect to its model. Alternatively, it can take a more conservative stance and eschew its model in favor of optimizing its behavior solely via real-world interaction. This latter approach can be exceedingly slow to learn from experience, while the former can lead to "planner overfitting" - aspects of the agent's behavior are optimized to exploit errors in its model. This paper explores an intermediate position in which the planner seeks to avoid overfitting through a kind of regularization of the plans it considers. We present three different approaches that demonstrably mitigate planner overfitting in reinforcement-learning environments.
Overcoming Overfitting in Reinforcement Learning via Gaussian Process Diffusion Policy
School of Electrical and Electronic Engineering, University of Sheffield(谢菲尔德大学电子与电气工程学院);Department of Electrical and Electronic Engineering, University of Manchester(曼彻斯特大学电子与电气工程系)
A Post-Training Approach for Mitigating Overfitting in Quantum Convolutional Neural Networks
University of Southern California(南加州大学);Viterbi School of Engineering(维特比工程学院);Information Sciences Institute(信息科学研究所);Department of Physics and Astronomy(物理与天文学系);Center for Quantum Information Science & Technology(量子信息科学与技术中心)
PRISMA: Reinforcement Learning Guided Two-Stage Policy Optimization in Multi-Agent Architecture for Open-Domain Multi-Hop Question Answering
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);The University of Western Australia(西澳大学);Huazhong University of Science and Technology(华中科技大学);Tsinghua University(清华大学);Shanghai AI Laboratory(上海人工智能实验室)
MoSA: Motion-constrained Stress Adaptation for Mitigating Real-to-Sim Gap in Continuum Dynamics via Learning Residual Anisotropy
Hong Kong University of Science(香港科学大学);MMLab, Chinese University of Hong Kong, Hong Kong SAR(香港中文大学MMLab, 香港特别行政区);The University of Hong Kong, Hong Kong SAR(香港大学, 香港特别行政区)