重新路由，而非移除：面向视觉语言模型的可恢复视觉令牌路由
Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models
AI总结：
针对视觉语言模型中视觉令牌重要性随解码器深度变化的问题，提出无需训练的可恢复路由方法Reroute，将不可逆移除改为可恢复路由，在激进令牌缩减下提升定位能力并保持通用VQA性能。
摘要：
视觉语言模型（VLM）将图像投影为数百到数千个视觉令牌，使得解码器推理在注意力计算和KV缓存内存方面代价高昂。现有的视觉令牌缩减方法大多遵循排序-移除范式：它们对视觉令牌进行评分，保留一个紧凑的子集，并永久丢弃其余部分。我们表明这种不可逆操作是脆弱的，因为视觉令牌的重要性随解码器深度变化；在某一阶段排名低的令牌可能在后续层中变得相关，尤其是对于需要定位的查询。我们提出Reroute，一种无需训练的插件，用可恢复路由替代移除。在每个路由阶段，选中的视觉令牌通过解码器块，而延迟的令牌绕过该阶段并在下一个路由决策时重新进入候选池。Reroute重用现有的注意力分数排序规则和阶段级调度，保留了它所增强的剪枝方法的理论TFLOPs和KV缓存预算类别。在LLaVA-1.5和Qwen骨干网络上的FastV、PDrop和Nüwa变体中，Reroute在激进令牌缩减下改善了定位性能，同时保持通用VQA性能。这些结果表明，VLM令牌缩减不应仅被视为不可逆剪枝，也应被视为可恢复路由。代码可在此处获取：this https URL
英文摘要：
Vision-language models (VLMs) project images into hundreds to thousands of visual tokens, making decoder inference expensive in both attention computation and KV-cache memory. Existing visual-token reduction methods largely follow a rank-and-remove paradigm: they score visual tokens, keep a compact subset, and permanently discard the rest. We show that this irreversible action is fragile because visual-token importance changes across decoder depth; tokens ranked low at one stage may become relevant in later layers, especially for grounding-sensitive queries. We propose Reroute, a training-free plug-in that replaces removal with recoverable routing. At each routing stage, selected vision tokens pass through decoder blocks, while deferred tokens bypass the stage and re-enter the candidate pool at the next routing decision. Reroute reuses existing attention-score ranking rules and stage-wise schedules, preserving the theoretical TFLOPs and KV-cache budget class of the pruning method it augments. Across FastV, PDrop, and Nüwa variants on LLaVA-1.5 and Qwen backbones, reroute improves grounding under aggressive token reduction while maintaining general VQA performance. These results suggest that VLM token reduction should not be viewed only as irreversible pruning, but also as recoverable routing. The code can be found here: https://github.com/elmma/mllm-reroute/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
