arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.08361cs.CV
收藏

$Δ$VLA:通过世界知识变化引导的视觉-语言-动作模型

$Δ$VLA: Prior-Guided Vision-Language-Action Models via World Knowledge Variation

Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Great Bay University(大湾大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Macao Polytechnic University(澳门理工学院)

Yijie Zhu, Jie He, Rui Shao, Kaishen Yuan, Tao Tan, Xiaochen Yuan, Zitong Yu

AI总结:

$Δ$VLA通过建模世界知识变化,结合先验引导和潜在空间学习,提升机器人动作生成的效率和性能。

摘要:

最近的视觉-语言-动作(VLA)模型通过统一感知、推理和控制显著推动了机器人操作的发展。为了实现这种整合,最近的研究采用了一种预测范式,通过建模未来的视觉状态或世界知识来指导动作生成。然而,这些模型强调预测结果而非推理变化过程的本质,这对于确定如何行动至关重要。为此,我们提出了$Δ$VLA,一种基于先验引导的框架,通过相对于显式当前世界知识先验建模世界知识变化来生成动作,而不是回归绝对的未来世界状态。具体而言,1)为了构建当前世界知识先验,我们提出了Prior-Guided WorldKnowledge Extractor(PWKE)。它从视觉输入中提取可操作区域、空间关系和语义线索,受辅助头和先验伪标签的引导,从而减少冗余。2)在此基础上,为了表示在动作下世界知识如何演变,我们引入了Latent World Variation Quantization(LWVQ)。它通过VQ-VAE目标学习离散的潜在空间来编码世界知识变化,将预测从全模态转移到紧凑的潜在空间。3)此外,为了在变化建模过程中减少干扰,我们设计了Conditional Variation Attention(CV-Atten),它促进了解耦学习并保持了知识表示的独立性。在模拟基准和真实世界机器人任务上的广泛实验表明,$Δ$VLA实现了最先进的性能,同时提高了效率。代码和真实世界执行视频可在https://github.com/JiuTian-VL/DeltaVLA上获得。

英文摘要:

Recent vision-language-action (VLA) models have significantly advanced robotic manipulation by unifying perception, reasoning, and control. To achieve such integration, recent studies adopt a predictive paradigm that models future visual states or world knowledge to guide action generation. However, these models emphasize forecasting outcomes rather than reasoning about the underlying process of change, which is essential for determining how to act. To address this, we propose $Δ$VLA, a prior-guided framework that models world-knowledge variations relative to an explicit current-world knowledge prior for action generation, rather than regressing absolute future world states. Specifically, 1) to construct the current world knowledge prior, we propose the Prior-Guided WorldKnowledge Extractor (PWKE). It extracts manipulable regions, spatial relations, and semantic cues from the visual input, guided by auxiliary heads and prior pseudo labels, thus reducing redundancy. 2) Building upon this, to represent how world knowledge evolves under actions, we introduce the Latent World Variation Quantization (LWVQ). It learns a discrete latent space via a VQ-VAE objective to encode world knowledge variations, shifting prediction from full modalities to compact latent. 3)Moreover, to mitigate interference during variation modeling, we design the Conditional Variation Attention (CV-Atten), whichpromotes disentangled learning and preserves the independence of knowledge representations. Extensive experiments on both simulated benchmarks and real-world robotic tasks demonstrate $Δ$VLA achieves state-of-the-art performance while improving efficiency. Code and real-world execution videos are available at https://github.com/JiuTian-VL/DeltaVLA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑