基于少量标注的面向对象世界模型用于样本高效强化学习
Object-Centric World Models from Few-Shot Annotations for Sample-Efficient Reinforcement Learning
AI总结：
本文提出OC-STORM框架，通过面向对象表示提升MBRL在复杂视觉领域的样本效率，实验证明其在Atari和Hollow Knight中的优越性能。
摘要：
尽管基于像素的深度强化学习（RL）已取得显著成功，但其样本低效性仍然是现实应用中的关键限制。基于模型的强化学习（MBRL）通过学习世界模型来生成模拟经验来解决这一问题，但依赖于像素级重建损失的标准方法往往无法捕捉复杂动态场景中的小而任务关键的对象。我们提出，面向对象（OC）的表示可以将模型能力定向于语义有意义的实体，从而提高动态预测和样本效率。在本文中，我们引入了OC-STORM，一种面向对象的MBRL框架，它通过预训练分割网络提取的对象表示来增强学习的世界模型。通过在极少量标注帧上进行条件化，OC-STORM能够学习跟踪决策相关对象的动力学和对象间相互作用，而无需大量标注或访问特权信息。实验证明，OC-STORM在Atari 100k基准测试中显著优于STORM基线，并在视觉复杂的Hollow Knight游戏中实现了最先进的样本效率。我们的发现强调了在复杂视觉领域将OC先验整合到MBRL中的潜力。项目页面：https://oc-storm.weipuzhang.com
英文摘要：
While deep reinforcement learning (RL) from pixels has achieved remarkable success, its sample inefficiency remains a critical limitation for real-world applications. Model-based RL (MBRL) addresses this by learning a world model to generate simulated experience, but standard approaches that rely on pixel-level reconstruction losses often fail to capture small, task-critical objects in complex, dynamic scenes. We posit that an object-centric (OC) representation can direct model capacity toward semantically meaningful entities, improving dynamics prediction and sample efficiency. In this work, we introduce OC-STORM, an object-centric MBRL framework that enhances a learned world model with object representations extracted by a pretrained segmentation network. By conditioning on a minimal number of annotated frames, OC-STORM learns to track decision-relevant object dynamics and inter-object interactions without extensive labeling or access to privileged information. Empirical results demonstrate that OC-STORM significantly outperforms the STORM baseline on the Atari 100k benchmark and achieves state-of-the-art sample efficiency on challenging boss fights in the visually complex game Hollow Knight. Our findings underscore the potential of integrating OC priors into MBRL for complex visual domains. Project page: https://oc-storm.weipuzhang.com
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
