State of the art reinforcement learning has enabled training agents on tasks of ever increasing complexity. However, the current paradigm tends to favor training agents from scratch on every new task or on collections of tasks with a view towards generalizing to novel task configurations. The former suffers from poor data efficiency while the latter is difficult when test tasks are out-of-distribution. Agents that can effectively transfer their knowledge about the world pose a potential solution to these issues. In this paper, we investigate transfer learning in the context of model-based agents. Specifically, we aim to understand when exactly environment models have an advantage and why. We find that a model-based approach outperforms controlled model-free baselines for transfer learning. Through ablations, we show that both the policy and dynamics model learnt through exploration matter for successful transfer. We demonstrate our results across three domains which vary in their requirements for transfer: in-distribution procedural (Crafter), in-distribution identical (RoboDesk), and out-of-distribution (Meta-World). Our results show that intrinsic exploration combined with environment models present a viable direction towards agents that are self-supervised and able to generalize to novel reward functions.
Enhancing Adversarial Transferability by Balancing Exploration and Exploitation with Gradient-Guided Sampling
School of Computer Science & Software Engineering, Shenzhen University, China（深圳大学计算机科学与软件工程学院）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Shenzhen, China（广东省人工智能与数字经济发展实验室（深圳））；Guangdong Provincial Key Laboratory of Intelligent Information Processing, Shenzhen University, China（广东省智能信息处理省级重点实验室）；School of Computer Science, University of Exeter, U.K.（埃克塞特大学计算机科学学院）；Department of Computing and Information Technology, Great Bay University, China（大鹏大学计算与信息技术系）；Computer Vision Institute, School of Artificial Intelligence, Shenzhen University, China（人工智能学院计算机视觉研究所）
Exploration in Knowledge Transfer Utilizing Reinforcement Learning
Institute of Information Theory and Automation, Czech Academy of Sciences（捷克科学院信息理论与自动化研究所）；Faculty of Applied Sciences, University of West Bohemia（西波希米亚大学应用科学学院）
InfoBot: Transfer and Exploration via the Information Bottleneck
Mila（米拉研究所）；University of Montreal（蒙特利尔大学）；McGill University（麦吉尔大学）；Deepmind（深度思维）；Google Brain（谷歌大脑）；University College London（伦敦大学学院）；University of California, Berkeley（加州大学伯克利分校）
Investigating layer-selective transfer learning of QAOA parameters for Max-Cut problem
Department of Engineering, University Pompeu Fabra（工程系，庞贝法布拉大学）；Department of Physics and Astronomy, University of Florence（物理与天文学系，佛罗伦萨大学）；QUANTIC, Barcelona Supercomputing Center（QUANTIC，巴塞罗那超级计算中心）；Institute for Cross-Disciplinary Physics and Complex Systems (IFISC) UIB-CSIC（跨学科物理与复杂系统研究所（IFISC） UIB-CSIC）；European Laboratory for Non-Linear Spectroscopy (LENS), University of Florence（非线性光谱欧洲实验室（LENS），佛罗伦萨大学）
