GRIP-VLM：基于组相对重要性的高效视觉-语言模型压缩
GRIP-VLM: Group-Relative Importance Pruning for Efficient Vision-Language Models
AI总结：
GRIP-VLM通过强化学习框架解决视觉语言模型中大规模视觉token处理的计算瓶颈，通过组相对重要性策略优化实现高效压缩，达到15%的推理加速。
摘要：
在视觉-语言模型（VLMs）中，处理大量视觉token会带来巨大的计算开销。尽管近期的训练感知剪枝方法试图选择性地丢弃冗余token，但它们主要依赖连续梯度放松。然而，视觉token剪枝本质上是一个离散、非凸的组合问题；因此，这些连续近似经常将优化困在次优局部极小值中，尤其是在激进压缩预算下。为克服这一根本瓶颈，我们提出了GRIP-VLM，一种由强化学习驱动的组相对重要性剪枝框架。不同于依赖平滑梯度假设，GRIP-VLM将剪枝建模为马尔可夫决策过程，采用由监督预热锚定的组相对策略优化（GRPO）范式，直接探索离散选择空间。结合预算感知评分器，我们的轻量级代理动态评估每个token的重要性，并适应任意压缩比例而无需重新训练。在多样化的多模态基准上进行的广泛实验表明，GRIP-VLM在性能上优于启发式和监督学习基线，实现了更优的帕累托前沿，并在等同准确性下达到高达15%的推理加速。
英文摘要：
In Vision-Language Models (VLMs), processing a massive number of visual tokens incurs prohibitive computational overhead. While recent training-aware pruning methods attempt to selectively discard redundant tokens, they largely rely on continuous-gradient relaxations. However, visual token pruning is inherently a discrete, non-convex combinatorial problem; consequently, these continuous approximations frequently trap the optimization in sub-optimal local minima, especially under aggressive compression budgets. To overcome this fundamental bottleneck, we propose GRIP-VLM, a Group-Relative Importance Pruning framework driven by Reinforcement Learning. Rather than relying on smooth-gradient assumptions, GRIP-VLM formulates pruning as a Markov Decision Process, employing a Group Relative Policy Optimization (GRPO) paradigm anchored by supervised warm-up to directly explore the discrete selection space. Integrated with a budget-aware scorer, our lightweight agent dynamically evaluates per-token importance and adapts to arbitrary compression ratios without retraining. Extensive experiments across diverse multimodal benchmarks demonstrate that GRIP-VLM consistently outperforms heuristic and supervised-learning baselines, achieving a superior Pareto frontier and delivering up to a 15\% inference speedup at equal accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
