arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.20566cs.ROcs.CV
收藏

BFA++: 多视角视觉语言动作模型的分层最佳特征感知令牌剪枝

BFA++: Hierarchical Best-Feature-Aware Token Prune for Multi-View Vision Language Action Model

Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);University of Chinese Academy of Sciences(中国科学院大学);LimX Dynamic(LimX动态);Nanjing University(南京大学);Nanyang Technological University(南洋理工大学);Zhejiang University(浙江大学)

Haosheng Li, Weixin Mao, Zihan Lan, Hongwei Xiong, Hongan Wang, Chenyang Si, Ziwei Liu, Xiaoming Deng, Hua Chen

AI总结:

BFA++通过分层最佳特征感知令牌剪枝提升多视角视觉语言动作模型的效率和成功率

摘要:

视觉语言动作(VLA)模型通过利用大视觉语言模型(VLMs)联合解释指令和视觉输入,已取得了显著突破。然而,视觉令牌数量的大幅增加,尤其是来自多视角输入的增加,对实时机器人操作构成了严重挑战。现有的VLM加速技术,如令牌剪枝,当直接应用于VLA模型时,往往会导致性能下降,因为它们忽略了不同视角之间的关系,并未考虑机器人操作的动态和任务特定特性。为此,我们提出了BFA++,一种专门针对VLA模型的动态令牌剪枝框架。BFA++引入了由两级重要性预测器引导的分层剪枝策略:一个内视角预测器在每张图像中突出显示与任务相关的区域以抑制空间噪声,而一个外视角预测器在不同的操作阶段识别关键的相机视角以减少跨视角冗余。这种设计使令牌选择变得高效,同时保留了必要的视觉线索,从而实现了更高的计算效率和更高的操作成功率。在RoboTwin基准和真实世界机器人任务上的评估表明,BFA++在两者上都优于现有方法。BFA++在π0和RDT模型上将成功率提高了约10%,分别实现了1.8倍和1.5倍的加速。我们的结果表明,上下文敏感且任务感知的令牌剪枝比完全视觉处理更有效,使实时机器人系统能够实现更快的推理和更高的操作准确性。

英文摘要:

Vision-Language-Action (VLA) models have achieved significant breakthroughs by leveraging Large Vision Language Models (VLMs) to jointly interpret instructions and visual inputs. However, the substantial increase in visual tokens, particularly from multi-view inputs, poses serious challenges to real-time robotic manipulation. Existing acceleration techniques for VLMs, such as token pruning, often result in degraded performance when directly applied to VLA models, as they overlook the relationships between different views and fail to account for the dynamic and task-specific characteristics of robotic operation. To address this, we propose BFA++, a dynamic token pruning framework designed specifically for VLA models. BFA++ introduces a hierarchical pruning strategy guided by two-level importance predictors: an intra-view predictor highlights task-relevant regions within each image to suppress spatial noise, while an inter-view predictor identifies critical camera views throughout different manipulation phases to reduce cross-view redundancy. This design enables efficient token selection while preserving essential visual cues, resulting in improved computational efficiency and higher manipulation success rates. Evaluations on the RoboTwin benchmark and real-world robotic tasks demonstrate that BFA++ consistently outperforms existing methods. BFA++ improves the success rate by about 10% on both the π0 and RDT models, achieving speedup of 1.8X and 1.5X, respectively. Our results highlight that context-sensitive and task-aware token pruning serves as a more effective strategy than full visual processing, enabling faster inference and improved manipulation accuracy in real-world robotic systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑