Meta-reinforcement learning (meta-RL) algorithms allow for agents to learn new behaviors from small amounts of experience, mitigating the sample inefficiency problem in RL. However, while meta-RL agents can adapt quickly to new tasks at test time after experiencing only a few trajectories, the meta-training process is still sample-inefficient. Prior works have found that in the multi-task RL setting, relabeling past transitions and thus sharing experience among tasks can improve sample efficiency and asymptotic performance. We apply this idea to the meta-RL setting and devise a new relabeling method called Hindsight Foresight Relabeling (HFR). We construct a relabeling distribution using the combination of "hindsight", which is used to relabel trajectories using reward functions from the training task distribution, and "foresight", which takes the relabeled trajectories and computes the utility of each trajectory for each task. HFR is easy to implement and readily compatible with existing meta-RL algorithms. We find that HFR improves performance when compared to other relabeling methods on a variety of meta-RL tasks.
Null Counterfactual Factor Interactions for Goal-Conditioned Reinforcement Learning
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of California San Diego（加州大学圣地亚哥分校）；MBZUAI（马克斯·普朗克人工智能研究所）；University of Massachusetts Amherst（马萨诸塞大学阿姆赫斯特分校）
本文针对目标条件强化学习在以对象为中心领域中的稀疏奖励问题，提出基于零反事实交互定义的 HInt 事后重新标记方法和 NCII 交互推断模型，显著提升交互推断准确率并将样本效率提高最多 4 倍。
Adaptable Hindsight Experience Replay for Search-Based Learning
Technical University of Darmstadt（德累斯顿技术大学）；Hessian Center for Artificial Intelligence（黑森人工智能中心）；German Research Center for Artificial Intelligence（德国人工智能研究中心）；Centre for Cognitive Science（认知科学中心）；National Research Center for Applied Cybersecurity ATHENE（应用网络安全国家研究中心ATHENE）
本文提出 Adaptable HER 框架，将事后经验回放与 AlphaZero 式搜索学习灵活集成，通过可调整的重新标记目标、策略目标和轨迹选择，在稀疏奖励及方程发现等任务中超越纯监督学习和强化学习。
Learning Stateful Predictive Knowledge From Experience
University College London（伦敦大学学院）；National University of Singapore（新加坡国立大学）；Chinese Academy of Sciences（中国科学院）；Brown University（布朗大学）；University of Bristol（布里斯托尔大学）；University of Edinburgh（爱丁堡大学）；Zhongguancun Academy（中关村科学院）；The Yangtze River Delta（长三角）
