arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2306.07307cs.LGcs.AI
收藏

面向少样本策略迁移的在线原型对齐

Online Prototype Alignment for Few-shot Policy Transfer

University of Science and Technology of China(中国科学技术大学);Institute of Computing Technology, CAS(中国科学院计算技术研究所);Cambricon Technologies(寒武纪科技);University of Chinese Academy of Sciences, China(中国科学院大学);Institute of Software Chinese Academy of Sciences(中国科学院软件研究所)

Qi Yi, Rui Zhang, Shaohui Peng, Jiaming Guo, Yunkai Gao, Kaizhao Yuan, Ruizhi Chen, Siming Lan, Xing Hu, Zidong Du, Xishan Zhang, Qi Guo, Yunji Chen

AI总结:

针对传统强化学习域适应方法依赖大量目标域数据、视觉差异大时易失效的问题,提出基于功能相似性的在线原型对齐框架,通过针对性探索关联跨域元素,实现少样本下更优的策略迁移性能。

摘要:

强化学习(RL)中的域适应主要解决策略迁移到新环境时的观测变化问题。许多传统的强化学习域适应方法试图以显式或隐式方式学习源域与目标域之间的映射函数,但它们通常需要获取目标域的大量数据。此外,这类方法往往依赖视觉线索学习映射函数,当源域与目标域外观差异较大时可能失效。为解决这些问题,我们提出了一种名为在线原型对齐(Online Prototype Alignment, OPA)的新型框架,该框架基于元素的功能相似性学习映射函数,仅需数个回合即可实现少样本策略迁移。OPA的核心思路是引入一种探索机制,能够高效且有针对性地与目标域的未见过元素交互,随后根据这些元素的功能(而非视觉线索)将其与源域中已见过的元素建立关联。实验结果表明,当目标域与源域视觉差异较大时,OPA即使使用少得多的目标域样本,也能取得更优的迁移性能,表现优于现有方法。

英文摘要:

Domain adaptation in reinforcement learning (RL) mainly deals with the changes of observation when transferring the policy to a new environment. Many traditional approaches of domain adaptation in RL manage to learn a mapping function between the source and target domain in explicit or implicit ways. However, they typically require access to abundant data from the target domain. Besides, they often rely on visual clues to learn the mapping function and may fail when the source domain looks quite different from the target domain. To address these problems, we propose a novel framework Online Prototype Alignment (OPA) to learn the mapping function based on the functional similarity of elements and is able to achieve the few-shot policy transfer within only several episodes. The key insight of OPA is to introduce an exploration mechanism that can interact with the unseen elements of the target domain in an efficient and purposeful manner, and then connect them with the seen elements in the source domain according to their functionalities (instead of visual clues). Experimental results show that when the target domain looks visually different from the source domain, OPA can achieve better transfer performance even with much fewer samples from the target domain, outperforming prior methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑