arXiv 2610.202232026-10-15cs.RO

ObjDream：基于物体级理解与未来状态预测的视觉-语言-动作模型引导框架

ObjDream: Guiding Vision-Language-Action Models with Object-Level Understanding and Future State Prediction

Southeast University（东南大学）；Shanghai Innovation Institute（上海创新研究院）；Shanghai Jiao Tong University（上海交通大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）

Huailiang Ma, Yuxuan Li, Ziyi Zhao, Xinkai Wang, Mingyu Li, Xinyu Guo, Lifeng Zhu, Aiguo Song, Cewu Lu, Jianhua Sun

备注：: 18 pages, 8 figures. Project page: https://huailiangma.github.io/objdream/

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

ObjDream是一种以物体级理解和未来状态预测为核心的VLA框架，通过时空物体编码器与动作条件物体动力学模块，提升了机器人操纵在分布偏移与场景扰动下的泛化能力，且可迁移至真实物理操纵场景。

摘要：

视觉-语言-动作（VLA）模型已成为机器人操纵领域的强大范式，可使机器人从视觉观测和语言指令中生成动作。然而，大多数现有VLA模型仍依赖预训练视觉-语言主干网络继承的图像级视觉表征，缺乏操纵所需的细粒度物体理解，导致其在分布外物体布局、杂乱场景及背景变化下鲁棒性下降。为解决这些局限，本文提出ObjDream，这是一个以物体级理解和未来物体状态预测为核心的以物体为中心的VLA框架。ObjDream采用时空物体编码器将视觉观测组织为时间物体槽，通过独立通道表征持久物体身份与动态物体状态；这些结构化物体表征被插入视觉-语言模型（VLM）的token序列中，身份引导路由机制可调节VLM对对应物体token的注意力。此外，动作条件物体动力学模块在训练期间预测与操纵相关的紧凑未来物体状态，明确监督机器人动作下的物体状态演化，推理阶段无需进行未来预测。在LIBERO、LIBERO-Plus和RoboTwin 2.0上的实验表明，ObjDream提升了在各类分布偏移和场景扰动下的泛化能力；真实世界机器人实验进一步证明，ObjDream可迁移至具有新颖物体排列和背景变化的物理操纵场景。

英文摘要：

Vision-language-action (VLA) models have emerged as a powerful paradigm for robotic manipulation, enabling robots to generate actions from visual observations and language instructions. However, most existing VLA models still operate on image-level visual representations inherited from pretrained vision-language backbones, while lacking fine-grained object understanding for manipulation. As a result, their robustness can degrade under out-of-distribution object layouts, clutter, and background variations. To address these limitations, we introduce ObjDream, an object-centric VLA framework centered on object-level understanding and future object state prediction. ObjDream uses a spatio-temporal object encoder to organize visual observations into temporal object slots, where persistent object identity and dynamic object state are represented through separate channels. These structured object representations are inserted into the token sequence of the vision-language model (VLM), where identity-guided routing modulates VLM attention to the corresponding object tokens. In addition, an action-conditioned object dynamics module predicts compact future object states relevant to manipulation during training, explicitly supervising object state evolution under robot actions without requiring future prediction at inference. Experiments on LIBERO, LIBERO-Plus, and RoboTwin 2.0 show that ObjDream improves generalization under diverse distribution shifts and scene perturbations. Real-world robot experiments further demonstrate that ObjDream transfers to physical manipulation settings with novel object arrangements and background changes.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。