广义非参数环境下的Meta-强化学习
Meta-Reinforcement Learning in Broad and Non-Parametric Environments
摘要:
最近最先进的人工智能体缺乏快速适应新任务的能力,因为它们专门针对特定目标进行训练,需要大量的互动来学习新技能。Meta-强化学习(meta-RL)通过利用从训练任务中学习到的知识在以前看不见的任务中表现良好来解决这一挑战。然而,目前的元强化学习方法局限于狭窄的参数任务分布,忽略了现实世界中发生的任务之间的质的差异。在本文中,我们介绍了TIGR,一个基于任务推理的元RL算法,使用高斯混合模型(GMM)和门控递归单元,设计用于非参数环境中的任务。我们采用了一个生成模型,涉及GMM捕捉多模态的任务。我们解耦的策略训练任务推理学习和有效的训练推理机制的基础上,无监督重建目标。我们提供了一个基于半猎豹环境的定性不同任务的基准测试,并证明了TIGR与最先进的元RL方法相比,在样本效率(快3-10倍),渐近性能和非参数环境中的适用性与zero-shot适应方面的卓越性能。
英文摘要:
Recent state-of-the-art artificial agents lack the ability to adapt rapidly to new tasks, as they are trained exclusively for specific objectives and require massive amounts of interaction to learn new skills. Meta-reinforcement learning (meta-RL) addresses this challenge by leveraging knowledge learned from training tasks to perform well in previously unseen tasks. However, current meta-RL approaches limit themselves to narrow parametric task distributions, ignoring qualitative differences between tasks that occur in the real world. In this paper, we introduce TIGR, a Task-Inference-based meta-RL algorithm using Gaussian mixture models (GMM) and gated Recurrent units, designed for tasks in non-parametric environments. We employ a generative model involving a GMM to capture the multi-modality of the tasks. We decouple the policy training from the task-inference learning and efficiently train the inference mechanism on the basis of an unsupervised reconstruction objective. We provide a benchmark with qualitatively distinct tasks based on the half-cheetah environment and demonstrate the superior performance of TIGR compared to state-of-the-art meta-RL approaches in terms of sample efficiency (3-10 times faster), asymptotic performance, and applicability in non-parametric environments with zero-shot adaptation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。