arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.34519cs.ROcs.AIcs.CVcs.LG
收藏

TRIAGE:高效的视觉-语言-动作推理需要同时兼顾复用与剪枝

TRIAGE: Efficient Vision-Language-Action Inference Needs Both Reuse and Pruning

Seoul National University(首尔国立大学);Carnegie Mellon University(卡内基梅隆大学)

Sooyoung Kwon, Carlee Joe-Wong, Kyunghan Lee

AI总结:

TRIAGE针对VLA模型在机载嵌入式板上的预填充延迟问题,结合令牌复用与剪枝策略,在保持高成功率的同时大幅降低延迟,在LIBERO等数据集上性能优于仅剪枝或仅复用的方法。

摘要:

视觉-语言-动作(VLA)模型因能将相机图像和语言指令直接映射为动作,正越来越多地应用于机器人操纵领域。对于无法连接到工作站的机器人(如移动操纵器和人形机器人),必须在机载嵌入式板上运行这些模型,而语言主干的预填充延迟会随每个控制步骤处理的数百个视觉令牌而增加。为满足实时性要求,无需训练的加速器仅重新计算部分视觉令牌,通过剪枝或复用其缓存的键值(KV)状态跳过其余令牌。这两种选择节省的计算量几乎相同,但现有方法对每个跳过的令牌仅采用其中一种方式。通过令牌级诊断,我们发现哪种选择能使动作更接近完整重新计算,取决于令牌的漂移(即自缓存计算以来其浅层值的变化)。在测试的两个主干上,复用漂移低于阈值的令牌的效果不劣于剪枝,而当漂移超过阈值时,复用产生的动作误差最高可达剪枝的4.4倍;相同的诊断可针对每个主干校准该阈值。基于这一发现,我们提出TRIAGE,其判定缓存有效的条件为:漂移低于该阈值且缓存不过旧。TRIAGE重新计算固定预算的令牌,优先处理缓存无效的重要令牌,复用所有缓存有效的其他令牌,剪枝其余令牌;仅剪枝和仅复用方法是该划分的两个极端。由于预算固定了重新计算的令牌数量,它直接决定了预填充延迟。在LIBERO和SimplerEnv上,TRIAGE的平均成功率与完整重新计算的差距在3个百分点以内,同时在Jetson AGX Orin上将预填充延迟降低了高达48%;仅重新计算7%的视觉令牌时,它在LIBERO-Object上比最强基线高出24.4个百分点。

英文摘要:

Vision-language-action (VLA) models are increasingly used for robot manipulation because they map camera images and a language instruction directly to actions. Robots that cannot be tethered to a workstation, such as mobile manipulators and humanoids, must run them on onboard embedded boards, where the prefill latency of the language backbone grows with the hundreds of visual tokens processed at every control step. To meet real-time limits, training-free accelerators recompute only a subset of the visual tokens and skip the rest, either by pruning them or by reusing their cached key-value (KV) states. The two options save nearly the same computation, yet existing methods apply only one of them to every skipped token. Through a token-level diagnostic, we show that which option keeps the action closer to full recomputation depends on the token's drift, the change in its shallow-layer value since its cache was computed. On both backbones tested, reusing tokens below a drift threshold was no worse than pruning them, whereas beyond it, reuse yields up to 4.4x the action error of pruning; the same diagnostic calibrates this threshold per backbone. Building on this finding, we propose TRIAGE, which deems a cache valid when its drift is below this threshold and it is not too old. TRIAGE recomputes a fixed budget of tokens, prioritizing important tokens with invalid caches, reuses every other token with a valid cache, and prunes the rest; prune-only and reuse-only methods are the two extremes of this partition. Because the budget fixes the number of recomputed tokens, it directly sets the prefill latency. On LIBERO and SimplerEnv, TRIAGE stays within 3 pp of full recomputation in average success rate while cutting prefill latency on a Jetson AGX Orin by up to 48%, and with only 7% of visual tokens recomputed, it beats the strongest baseline by 24.4 pp on LIBERO-Object.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑