arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2611.06447cs.RO
收藏

$A^3$-VLA:面向机器人操作的自动感知引导注意力对齐

$A^3$-VLA: Automatic Perception-Guided Attention Alignment for Robot Manipulation

Shanghai AI Laboratory(上海人工智能实验室);College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(南京航空航天大学计算机科学与技术学院);School of Artificial Intelligence and Data Science, University of Science and Technology of China(中国科学技术大学人工智能与数据科学学院);State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China(中国科学技术大学认知智能国家重点实验室);Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)人工智能学域);Department of Computer Science and Engineering, The Hong Kong University of Science and Technology(香港科技大学计算机科学与工程系)

Shengzhe Zhang, Qi Zhang, Dazhong Shen, Chao Wang, Shaopeng Zhai, Hui Xiong

AI总结:

$A^3$-VLA框架针对VLAs注意力偏离问题,通过生成并优化候选区域作为感知引导,结合注意力分配与对比学习损失,提升了多种VLA骨干模型的操作任务成功率。

摘要:

视觉-语言-动作模型(VLAs)通过利用视觉-语言模型(VLMs)处理视觉观测和语言指令,使机器人能够执行操作任务。然而,动作预测目标本身缺乏细粒度的特定任务感知监督,这使得它容易依赖与任务无关的视觉捷径,导致注意力偏离目标物体和关键交互区域,从而限制了其操作性能和泛化能力。为解决该问题,我们提出面向机器人操作的自动感知引导注意力对齐框架($A^3$-VLA),该框架从操作过程本身推导特定任务的视觉引导。具体而言,$A^3$-VLA首先从交互诱导的视觉变化中生成目标物体和机器人的启发式候选区域,并通过视觉分割模型对其进行离线优化。这些区域通过两个互补目标作为显式感知引导:一是注意力分配损失,当语言或机器人状态标记查询视觉输入时,该损失鼓励对任务关键区域的充分关注;二是对比学习损失,该损失使候选目标区域的表示与语言指令对齐,同时将其与任务无关区域分离。在模拟和真实操作任务上开展的大量实验表明,$A^3$-VLA可持续提升多种VLA骨干模型的性能,在各基准测试中平均成功率提升幅度为6.5%至25.2%。

英文摘要:

Vision-Language-Action Models (VLAs) enable robots to perform manipulation tasks by leveraging Vision-Language Models (VLMs) to process visual observations and language instructions. However, the action prediction objective itself lacks fine-grained task-specific perceptual supervision, making it prone to relying on task-irrelevant visual shortcuts, leading attention away from target objects and key interaction areas, thereby limiting its operational performance and generalization ability. To address this, we propose Automatic Perception-Guided Attention Alignment for Robot Manipulation ($A^3$-VLA), a framework that derives task-specific visual guidance from the manipulation process itself. Specifically, $A^3$-VLA first generates heuristic candidate regions for target objects and the robot from interaction-induced visual changes, and refines them with a visual segmentation model offline. These regions serve as explicit perceptual guidance through two complementary objectives: an attention allocation loss that encourages sufficient focus on task-critical regions when language or robot state tokens query the visual input, and a contrastive learning loss that aligns the representations of candidate target regions with language instructions while separating them from task-irrelevant regions. Extensive experiments on simulated and real-world manipulation tasks demonstrate that $A^3$-VLA consistently improves multiple VLA backbones, with an average success rate gain of 6.5% - 25.2% across benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑