TRIO: 通过推理目标引导的令牌缩减以提高视觉-语言模型的效率
TRIO: Token Reduction via Inference-Objective Guidance for Efficient Vision-Language Models
AI总结：
TRIO从推理目标角度出发，通过梯度显著性指导视觉令牌压缩，保留输出不变性，提升视觉-语言模型的推理效率。
摘要：
最近，减少视觉语言模型（VLMs）中的冗余视觉令牌以加速推理已成为热门话题。然而，大多数现有方法依赖于基于视觉令牌间相似性或跨模态视觉-文本相似性的启发式方法，导致压缩性能和实际部署存在限制。相比之下，我们从推理目标的角度提出TRIO，将视觉令牌压缩转化为保持输出不变性，通过其对这一目标的重要性选择令牌。具体而言，通过我们设计的层局部代理损失生成的令牌级梯度显著性指导视觉令牌重新排序，然后根据非最大抑制（NMS）原则选择最有价值的视觉令牌。所提出的TRIO是训练免费的，并且兼容FlashAttention，适用于实际应用和部署。它可以作为无编码器的方法独立部署，或与编码器压缩方法如VisionZip结合使用作为有编码器的方法。在LLaVA-Next-7B上，TRIO仅保留11.1%的视觉令牌，但保持97.2%的原始性能，预填充速度提升2.75倍，推理速度提升2.14倍，FLOPs降低6.22倍，KV缓存开销减少6.05倍。我们的代码可在https://github.com/ocy1/TRIO上获得。
英文摘要：
Recently, reducing redundant visual tokens in vision-language models (VLMs) to accelerate VLM inference has emerged as a hot topic. However, most existing methods rely on heuristics constructed based on inter-visual-token similarity or cross-modal visual-text similarity, which gives rise to certain limitations in compression performance and practical deployment. In contrast, we propose TRIO from the perspective of inference objectives, which transforms visual token compression into preserving output result invariance and selects tokens primarily by their importance to this goal. Specifically, vision tokens are reordered with the guidance of token-level gradient saliency generated by our designed layer-local proxy loss, a coarse constraint from the current layer to the final result. Then the most valuable vision tokens are selected following the non-maximum suppression (NMS) principle.The proposed TRIO is training-free and compatible with FlashAttention, friendly to practical application and deployment. It can be deployed independently as an encoder-free method, or combined with encoder compression approaches like VisionZip for use as an encoder-involved method. On LLaVA-Next-7B, TRIO retains just 11.1\% of visual tokens but maintains 97.2\% of the original performance, with a 2.75$\times$ prefill speedup, 2.14$\times$ inference speedup, 6.22$\times$ lower FLOPs, and 6.05$\times$ reduced KV Cache overhead.Our code is available at https://github.com/ocy1/TRIO.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
