arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.12845cs.AIcs.GT
收藏

MORE-3S:基于共享语义空间的多模态离线强化学习

MORE-3S:Multimodal-based Offline Reinforcement Learning with Shared Semantic Spaces

Beijing University of Posts and Telecommunications(北京邮电大学);University of Waterloo(滑铁卢大学)

Tianyu Zheng, Ge Zhang, Xingwei Qu, Ming Kuang, Stephen W. Huang, Zhaofeng He

AI总结:

本文提出MORE-3S,通过将图像状态与文本动作对齐到共享语义空间,把离线强化学习转化为有监督学习任务,在Atari和OpenAI Gym上显著超越现有基线。

摘要:

基于将不同模态对齐到同一语义嵌入空间可使模型更容易理解状态和动作这一直觉,我们为离线强化学习(RL)挑战提出了一种新视角。更具体地说,我们通过整合多模态和预训练语言模型,将其转化为一个有监督学习任务。我们的方法融合了从图像中获取的状态信息和从文本中获取的动作相关数据,从而提升强化学习训练性能并促进长期策略性思考。我们强调语言的上下文理解,并展示了强化学习中的决策如何受益于将状态和动作的表示与语言表示对齐。在Atari和OpenAI Gym环境上进行的评估表明,我们的方法显著优于现有基线。这有助于提升离线强化学习的性能和效率,同时为离线强化学习提供了一种新颖的视角。我们的代码和数据可在https://github.com/Zheng0428/MORE_获取。

英文摘要:

Drawing upon the intuition that aligning different modalities to the same semantic embedding space would allow models to understand states and actions more easily, we propose a new perspective to the offline reinforcement learning (RL) challenge. More concretely, we transform it into a supervised learning task by integrating multimodal and pre-trained language models. Our approach incorporates state information derived from images and action-related data obtained from text, thereby bolstering RL training performance and promoting long-term strategic thinking. We emphasize the contextual understanding of language and demonstrate how decision-making in RL can benefit from aligning states' and actions' representation with languages' representation. Our method significantly outperforms current baselines as evidenced by evaluations conducted on Atari and OpenAI Gym environments. This contributes to advancing offline RL performance and efficiency while providing a novel perspective on offline RL.Our code and data are available at https://github.com/Zheng0428/MORE_.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑