arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.19320cs.LGcs.AIcs.CV
收藏

具有上下文感知标记化的高效世界模型

Efficient World Models with Context-Aware Tokenization

University of Geneva(日内瓦大学)

Vincent Micheli, Eloi Alonso, François Fleuret

AI总结:

本文提出新智能体$Δ$-IRIS,通过离散自编码器编码时间步随机增量与自回归Transformer预测未来增量的世界模型架构,在Crafter基准上以更少帧预算达到最优且训练速度提升一个数量级。

摘要:

扩大深度强化学习(RL)方法的规模面临重大挑战。随着生成式建模的发展,基于模型的RL成为强有力的竞争者。序列建模的最新进展催生了有效的基于Transformer的世界模型,但代价是由于准确模拟环境需要长序列标记,导致计算量繁重。在这项工作中,我们提出了$Δ$-IRIS,一种新智能体,其世界模型架构由一个编码时间步之间随机增量的离散自编码器,和一个通过用连续标记概括世界当前状态来预测未来增量的自回归Transformer组成。在Crafter基准上,$Δ$-IRIS在多个帧预算下达到了新的最优水平,同时训练速度比以往基于注意力的方法快一个数量级。我们在https://github.com/vmicheli/delta-iris发布了代码和模型。

英文摘要:

Scaling up deep Reinforcement Learning (RL) methods presents a significant challenge. Following developments in generative modelling, model-based RL positions itself as a strong contender. Recent advances in sequence modelling have led to effective transformer-based world models, albeit at the price of heavy computations due to the long sequences of tokens required to accurately simulate environments. In this work, we propose $Δ$-IRIS, a new agent with a world model architecture composed of a discrete autoencoder that encodes stochastic deltas between time steps and an autoregressive transformer that predicts future deltas by summarizing the current state of the world with continuous tokens. In the Crafter benchmark, $Δ$-IRIS sets a new state of the art at multiple frame budgets, while being an order of magnitude faster to train than previous attention-based approaches. We release our code and models at https://github.com/vmicheli/delta-iris.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑