面向语言引导操作的对象中心世界模型
Object-Centric World Model for Language-Guided Manipulation
AI总结：
本文提出一种基于槽注意力的对象中心世界模型，通过语言指令在紧凑表示空间中预测未来状态，在视觉-语言-运动控制任务中比生成式世界模型具有更高的样本与计算效率。
摘要：
世界模型对于智能体在自动驾驶和机器人等领域中预测未来并进行规划至关重要。为实现这一目标，近期研究进展集中在视频生成上，由于扩散模型的显著成功，该方法受到了广泛关注。然而，这些模型需要大量的计算资源。为应对这些挑战，我们提出了一种利用槽注意力（slot attention）的对象中心表示空间、并由语言指令引导的世界模型。我们的模型将当前状态感知为对象中心表示，并在该表示空间中基于自然语言指令预测未来状态。与基于扩散的生成式替代方案相比，该方法得到了更紧凑且计算效率更高的模型。此外，它能够根据语言指令灵活预测未来状态，并在对象识别至关重要的操作任务中具有显著优势。在本文中，我们证明了所提出的潜在预测世界模型在视觉-语言-运动控制任务中优于生成式世界模型，取得了更高的样本效率和计算效率。我们还研究了所提方法的泛化性能，并探索了利用对象中心表示预测动作的各种策略。
英文摘要：
A world model is essential for an agent to predict the future and plan in domains such as autonomous driving and robotics. To achieve this, recent advancements have focused on video generation, which has gained significant attention due to the impressive success of diffusion models. However, these models require substantial computational resources. To address these challenges, we propose a world model leveraging object-centric representation space using slot attention, guided by language instructions. Our model perceives the current state as an object-centric representation and predicts future states in this representation space conditioned on natural language instructions. This approach results in a more compact and computationally efficient model compared to diffusion-based generative alternatives. Furthermore, it flexibly predicts future states based on language instructions, and offers a significant advantage in manipulation tasks where object recognition is crucial. In this paper, we demonstrate that our latent predictive world model surpasses generative world models in visuo-linguo-motor control tasks, achieving superior sample and computation efficiency. We also investigate the generalization performance of the proposed method and explore various strategies for predicting actions using object-centric representations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
