arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.17032cs.ROcs.CVcs.LG
收藏

M2CURL:面向机器人操作的基于自监督表示学习的样本高效多模态强化学习

M2CURL: Sample-Efficient Multimodal Reinforcement Learning via Self-Supervised Representation Learning for Robotic Manipulation

University of Leoben(莱奥本大学)

Fotios Lygerakis, Vedant Dave, Elmar Rueckert

AI总结:

针对多模态强化学习中视觉-触觉数据表示学习难、样本效率低的问题,提出M2CURL框架,通过多模态自监督对比学习学习高效表示,可适配任意强化学习算法,在Tactile Gym 2的操作任务中显著提升收敛速度和累积奖励。

摘要:

多模态强化学习(RL)最关键的问题之一是不同观测模态的有效融合。从这些模态中得到鲁棒且准确的表示,是提升强化学习算法鲁棒性与样本效率的核心。然而,在强化学习场景下学习视觉-触觉数据的表示面临巨大挑战,主要源于数据的高维度,以及将视觉、触觉输入与动态环境、任务目标建立关联的复杂性。为解决这些挑战,我们提出多模态对比无监督强化学习(Multimodal Contrastive Unsupervised Reinforcement Learning,M2CURL)。该方法采用一种新颖的多模态自监督学习技术,能够学习高效的表示,加快强化学习算法的收敛速度。我们的方法与具体强化学习算法无关,因此可以与任意现有强化学习算法结合。我们在Tactile Gym 2模拟器上对M2CURL进行了评估,结果表明它在不同操作任务中显著提升了学习效率。与未使用我们的表示学习方法的标准强化学习算法相比,该方法收敛速度更快,每回合累积奖励更高。

英文摘要:

One of the most critical aspects of multimodal Reinforcement Learning (RL) is the effective integration of different observation modalities. Having robust and accurate representations derived from these modalities is key to enhancing the robustness and sample efficiency of RL algorithms. However, learning representations in RL settings for visuotactile data poses significant challenges, particularly due to the high dimensionality of the data and the complexity involved in correlating visual and tactile inputs with the dynamic environment and task objectives. To address these challenges, we propose Multimodal Contrastive Unsupervised Reinforcement Learning (M2CURL). Our approach employs a novel multimodal self-supervised learning technique that learns efficient representations and contributes to faster convergence of RL algorithms. Our method is agnostic to the RL algorithm, thus enabling its integration with any available RL algorithm. We evaluate M2CURL on the Tactile Gym 2 simulator and we show that it significantly enhances the learning efficiency in different manipulation tasks. This is evidenced by faster convergence rates and higher cumulative rewards per episode, compared to standard RL algorithms without our representation learning approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑