arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1909.02291cs.LGcs.AI
收藏

具有动作嵌入的可迁移学习策略

Learning Action-Transferable Policy with Action Embedding

NetEase Fuxi AI Lab(网易伏羲人工智能实验室);Zhejiang University(浙江大学);College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室)

Yu Chen, Yingfeng Chen, Zhipeng Hu, Tianpei Yang, Changjie Fan, Yang Yu, Jianye Hao

摘要:

迁移学习(TL)是提高强化学习样本效率的有效方法。然而,如何有效地转移知识的任务与不同的状态-动作空间的研究在早期阶段。以往的研究大多只通过学习一个共同的特征空间来解决不同状态空间之间的不一致性,而没有考虑到相关任务的不同动作空间中的相似动作具有相似的语义。在本文中,我们提出了一种利用这一思想学习动作嵌入的方法,以及一个学习状态嵌入和动作嵌入的框架,以在具有不同状态和动作空间的任务之间传输策略。实验结果表明,该方法不仅可以学习信息丰富的动作嵌入,而且可以加速策略学习。

英文摘要:

Transfer learning (TL) is a promising way to improve the sample efficiency of reinforcement learning. However, how to efficiently transfer knowledge across tasks with different state-action spaces is investigated at an early stage. Most previous studies only addressed the inconsistency across different state spaces by learning a common feature space, without considering that similar actions in different action spaces of related tasks share similar semantics. In this paper, we propose a method to learning action embeddings by leveraging this idea, and a framework that learns both state embeddings and action embeddings to transfer policy across tasks with different state and action spaces. Our experimental results on various tasks show that the proposed method can not only learn informative action embeddings but accelerate policy learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑