MI-Pruner：跨模态互信息引导的令牌修剪器用于高效的大语言模型
MI-Pruner: Crossmodal Mutual Information-guided Token Pruner for Efficient MLLMs
AI总结：
本文提出MI-Pruner，通过计算视觉与文本特征间的互信息来指导令牌修剪，无需依赖注意力机制，实现高效的大语言模型推理。
摘要：
对于多模态大语言模型（MLLMs），视觉信息相比文本较为稀疏。因此，研究视觉修剪以提高推理效率成为必要。当前方法通常基于视觉编码器或LLM解码器中的注意力分数测量令牌重要性，然后选择高注意力分数的视觉令牌进行修剪。本文提出不同的更精细的方法。不依赖机制特定的信号，直接计算视觉与文本特征之间的互信息（MI），在它们交互之前。这允许我们在特征层面显式测量跨模态依赖性。我们的MI-Pruner简单、高效且非侵入性，无需访问内部注意力图或架构修改。实验结果表明，我们的方法在最小延迟下优于先前基于注意力的修剪方法。
英文摘要：
For multimodal large language models (MLLMs), visual information is relatively sparse compared with text. As a result, research on visual pruning emerges for efficient inference. Current approaches typically measure token importance based on the attention scores in the visual encoder or in the LLM decoder, then select visual tokens with high attention scores while pruning others. In this paper, we pursue a different and more surgical approach. Instead of relying on mechanism-specific signals, we directly compute Mutual Information (MI) between visual and textual features themselves, prior to their interaction. This allows us to explicitly measure crossmodal dependency at the feature levels. Our MI-Pruner is simple, efficient and non-intrusive, requiring no access to internal attention maps or architectural modifications. Experimental results demonstrate that our approach outperforms previous attention-based pruning methods with minimal latency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
