arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.14622cs.LGcs.AI
收藏

ACT-JEPA:用于高效策略表征学习的新型联合嵌入预测架构

ACT-JEPA: Novel Joint-Embedding Predictive Architecture for Efficient Policy Representation Learning

Faculty of Technical Sciences, University of Novi Sad(技术科学学院,诺维萨德大学)

Aleksandar Vujinovic, Aleksandar Kovacevic

AI总结:

提出ACT-JEPA架构,结合模仿学习与自监督学习,通过联合预测动作序列和潜在观测序列学习鲁棒世界模型,在多个环境中任务成功率提升达10%。

摘要:

学习决策策略的高效表征是模仿学习(IL)中的一个挑战。当前的IL方法需要专家演示,而收集这些演示成本高昂。此外,它们并未明确训练以理解环境,因此世界模型发展不足。自监督学习(SSL)提供了一种替代方案,因为它可以从多样化的未标记数据中学习世界模型。然而,大多数SSL方法效率低下,因为它们在原始输入空间中操作。在这项工作中,我们提出了ACT-JEPA,一种新颖的架构,它统一了IL和SSL以增强策略表征。它被端到端训练以联合预测1)动作序列和2)潜在观测序列。为了在潜在空间中学习,我们利用了联合嵌入预测架构,该架构允许模型过滤掉不相关的细节并学习鲁棒的世界模型。我们在不同环境和多个任务中评估了ACT-JEPA。我们的结果表明,它在所有环境中都优于最强的基线。ACT-JEPA在世界模型理解上实现了高达40%的提升,任务成功率提高了10%。最后,我们表明预测潜在观测序列有效地泛化到预测动作序列。这项工作展示了如何将IL和SSL集成以实现高效策略表征学习、改进的世界模型和更高的任务成功率。

英文摘要:

Learning efficient representations for decision-making policies is a challenge in imitation learning (IL). Current IL methods require expert demonstrations, which are expensive to collect. Additionally, they are not explicitly trained to understand the environment. Consequently, they have underdeveloped world models. Self-supervised learning (SSL) offers an alternative, as it can learn a world model from diverse, unlabeled data. However, most SSL methods are inefficient because they operate in raw input space. In this work, we propose ACT-JEPA, a novel architecture that unifies IL and SSL to enhance policy representations. It is trained end-to-end to jointly predict 1) action sequences and 2) latent observation sequences. To learn in latent space, we utilize Joint-Embedding Predictive Architecture, which allows the model to filter out irrelevant details and learn a robust world model. We evaluate ACT-JEPA in different environments and across multiple tasks. Our results show that it outperforms the strongest baseline in all environments. ACT-JEPA achieves up to 40% improvement in world model understanding and up to 10% higher task success rate. Finally, we show that predicting latent observation sequences effectively generalizes to predicting action sequences. This work demonstrates how integrating IL and SSL leads to efficient policy representation learning, an improved world model, and a higher task success rate.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑