ContraBAR：对比式贝叶斯自适应深度强化学习
ContraBAR: Contrastive Bayes-Adaptive Deep RL
AI总结：
提出 ContraBAR 元强化学习算法，用对比预测编码 CPC 替代变分信念推断，并证明其表征足以实现贝叶斯最优性，可在线上、离线及图像观测场景中取得先进性能。
摘要：
在元强化学习（meta RL）中，智能体寻求一种贝叶斯最优策略——即面对从某个已知任务分布中采样的未知任务时的最优策略。以往方法通过使用变分推断方法推断任务参数上的信念来解决这一问题。受对比学习方法近期在强化学习中取得成功的启发，例如对比预测编码（CPC），我们研究了对比方法是否可用于学习贝叶斯最优行为。我们首先证明了由 CPC 学习到的表征确实足以实现贝叶斯最优性。基于这一观察，我们提出了一种简单的元强化学习算法，该算法使用 CPC 代替变分信念推断。我们的方法 ContraBAR 在基于状态的观测领域中取得了与最先进方法相当的性能，并规避了未来观测重建的计算代价，从而能够在基于图像的观测领域中进行学习。它还可以与用于领域随机化的图像增强相结合，并能无缝用于在线和离线元强化学习设置。
英文摘要：
In meta reinforcement learning (meta RL), an agent seeks a Bayes-optimal policy -- the optimal policy when facing an unknown task that is sampled from some known task distribution. Previous approaches tackled this problem by inferring a belief over task parameters, using variational inference methods. Motivated by recent successes of contrastive learning approaches in RL, such as contrastive predictive coding (CPC), we investigate whether contrastive methods can be used for learning Bayes-optimal behavior. We begin by proving that representations learned by CPC are indeed sufficient for Bayes optimality. Based on this observation, we propose a simple meta RL algorithm that uses CPC in lieu of variational belief inference. Our method, ContraBAR, achieves comparable performance to state-of-the-art in domains with state-based observation and circumvents the computational toll of future observation reconstruction, enabling learning in domains with image-based observations. It can also be combined with image augmentations for domain randomization and used seamlessly in both online and offline meta RL settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
